Data query method, device, apparatus and medium

Abstract

This application provides a data query method, apparatus, device, and medium. By dividing the query term sequence into a first query term and a second query term, and determining the corresponding semantic nearest neighbor set and query condition probability for each second query term based on its prefix in the query term sequence, and combining the text content of the first query term, the query term sequence is matched with each stored term sequence indicated by the target multi-way tree. This significantly improves the hit rate and reuse rate of cached key-value vectors, effectively reuses key-value vectors of terms with the same or similar semantics but different expressions, and significantly reduces memory usage while ensuring generation quality and cache hit rate. This improves the overall efficiency and scalability of large language model inference, and achieves synergistic optimization of large language models in three dimensions: memory efficiency, throughput performance, and generation quality. It effectively reduces the problems of low cache hit rate, serious memory redundancy, and unstable generation quality in related technologies.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a data query method, apparatus, device, and medium. Background Technology

[0002] Key-value caching (KV Cache) is a crucial technique in autoregressive inference of Large Language Models (LLMs). During inference, LLMs generate new lexical units based on the input lexical sequence. These new lexical units are then concatenated with the old lexical sequence as new input, and this process is repeated to generate new lexical units until the output lexical sequence is obtained. Each inference iteration requires converting each lexical unit into three feature vectors: a query (Q) vector, an attention key (K) vector, and a value (V) vector. These vectors are then used to perform calculations based on the Q vector of the new lexical unit and the K and V vectors of the old lexical units. By caching the K and V vectors of historical lexical units, these vectors can be reused, avoiding redundant calculations and significantly improving generation efficiency.

[0003] To reuse cached key-value vectors, most methods use exact prefix sharing. Only when the token sequences of multiple requests are exactly the same, the key-value vectors are reused. Although the above method can achieve cache hits, exact prefix sharing only supports exact literal matching, which has great limitations. This results in the redundant storage of reusable key-value vectors, leading to serious memory waste. Summary of the Invention

[0004] In view of this, this application provides a data query method, apparatus, device and medium that constructs semantic nearest neighbors of query terms based on prefixes in the query term sequence, and realizes matching based on the context. It can effectively reuse key-value vectors of stored terms with the same semantics but different expressions, and improve cache hit rate while ensuring matching quality.

[0005] Specifically, this application is implemented through the following technical solution: According to a first aspect of this application, a data query method is provided, the method comprising: Based on a first preset number, multiple query words in the query word sequence are divided into a first query word and a second query word; the first query word is the first preset number of query words in the query word sequence, and the second query word is the other query words in the query word sequence besides the first query word; the query word sequence is the initial prompt word input for the large language model; For each second query term, based on the prefix of the second query term in the query term sequence, determine the query semantic nearest neighbor set and query condition probability corresponding to the second query term; Based on the text content of the first query term, the query semantic nearest neighbor set of the second query term, and the query condition probability, the query term sequence is matched with each stored term sequence indicated by the target multi-way tree to obtain a matching result; in the target multi-way tree, the root node is empty, and each other node except the root node is used to store a corresponding stored term and the key value vector position information of the stored term. The stored terms from the root node to any other node constitute a stored term sequence, and the common stored terms of different stored term sequences are stored once in the target multi-way tree; When the matching result indicates the existence of a target stored word sequence that matches the query word sequence, the target key value vector of each target stored word is obtained based on the key value vector position information of each target stored word in the target stored word sequence; the target key value vector is used to generate the target prompt word input of the large language model.

[0006] In one optional implementation, determining the query semantic nearest neighbor set and query condition probability corresponding to the second query term based on its prefix in the query term sequence includes: The prefix of the second query term in the query term sequence is input into the large language model to obtain the conditional probability distribution corresponding to the second query term output by the large language model; the conditional probability distribution is used to indicate the probability value of each preset term in the model vocabulary occurring at the position of the second query term under the condition that the prefix of the query term occurs; the model vocabulary is used to record the preset terms that the large language model can output. Based on the probability value corresponding to each preset word element, the second preset number of preset word elements with the highest probability value are determined from the model vocabulary and identified as the target preset word element. The set of target preset words is determined as the query semantic nearest neighbor set corresponding to the second query word; Based on the probability value corresponding to the target preset word, the query condition probability corresponding to the second query word is determined.

[0007] In one optional implementation, in the target multi-way tree, each node with a hierarchical interval greater than the first preset number of nodes is also used to store the storage semantic nearest neighbor set and storage conditional probability of the corresponding storage word. The process involves matching the query term sequence with each stored term sequence indicated by the target multi-way tree based on the text content of the first query term, the query semantic nearest neighbor set of the second query term, and the query conditional probability, to obtain a matching result, including: According to the hierarchical order of the target multi-branch tree, based on the text content of the first query term and the text content of the first stored term in each stored term sequence indicated by the target multi-branch tree, the first query term is sequentially matched with the first stored term in each stored term sequence to obtain the text matching result; the first stored term is the first preset number of stored terms in the stored term sequence. If the text matching result indicates the existence of a candidate stored word sequence that matches the query word sequence text, for each candidate stored word sequence that matches the text, based on the query semantic nearest neighbor set and the query conditional probability of the second query word, and the storage semantic nearest neighbor set and the storage conditional probability of the second stored word, the second query word is sequentially semantically matched with the second stored word in the candidate stored word sequence until the last second query word in the query word sequence is semantically matched successfully or any second query word fails to be semantically matched, thus obtaining a semantic matching result; the second stored word is any other stored word in the stored word sequence other than the first stored word. The candidate stored word sequence indicated by the semantic matching result is determined as the target stored word sequence that matches the query word sequence; the sequence length of the target stored word sequence is less than or equal to the sequence length of the query word sequence.

[0008] In one optional implementation, the semantic match of any of the second query terms is determined to be successful through the following steps: When the second query term belongs to the storage semantic nearest neighbor set of the second stored term, or when the second stored term belongs to the query semantic nearest neighbor set of the second query term, the query condition probability and the storage condition probability are compared with the probability threshold respectively. If both the query condition probability and the storage condition probability are greater than or equal to the probability threshold, the second query term semantic match is determined to be successful.

[0009] In one optional implementation, the method further includes: If the sequence length of the target stored word sequence is less than the sequence length of the query word sequence, extract the unmatched query word sequence relative to the target stored word sequence from the query word sequence; The node corresponding to the last stored word in the target stored word sequence is determined as the insertion node; In the target multi-way tree, after the insertion node, corresponding nodes are created for each query term in the sequence of unmatched query terms.

[0010] In one optional implementation, the method further includes: Construct an undirected graph corresponding to the target multi-way tree; the undirected graph includes a sequence set and an edge set, the sequence set includes the stored word sequence indicated by the target multi-way tree, and the edge set includes the edge formed by connecting two stored word sequences with semantic similarity greater than or equal to a similarity threshold; A depth-first search is performed on the undirected graph to extract multiple connected components; each connected component includes multiple semantically similar stored word sequences. For each connected component, the representative stored word sequence with the highest number of successful matches is determined from the multiple stored word sequences included in the connected component; For each other stored word sequence in the connected component other than the representative stored word sequence, match the other stored word sequence with the representative stored word sequence; For the successfully matched stored lexical units in the other stored lexical unit sequences, the key-value vector position information of the node corresponding to the matched stored lexical unit is adjusted to the key-value vector position information of the corresponding matched stored lexical unit in the representative stored lexical unit sequence, and the key-value vector of the matched stored lexical unit is deleted.

[0011] In one alternative implementation, the semantic similarity between two stored lexical sequences is determined through the following steps: Determine the sequence length difference, maximum sequence length, and minimum sequence length between two stored word sequences; Based on the maximum sequence length and length tolerance, determine the length difference threshold; If the sequence length difference is less than or equal to the length difference threshold, the semantic similarity between two stored word sequences is determined based on the stored semantic nearest neighbor set of stored words in each stored word sequence and the minimum sequence length. If the sequence length difference is greater than the length difference threshold, the semantic similarity between the two stored word sequences is determined to be 0.

[0012] According to a second aspect of this application, a data query apparatus is provided, the apparatus comprising: The lexical classification module is used to divide multiple query lexical units in the query lexical unit sequence into a first query lexical unit and a second query lexical unit based on a first preset number; the first query lexical unit is the first preset number of query lexical units in the query lexical unit sequence, and the second query lexical unit is the other query lexical units in the query lexical unit sequence besides the first query lexical unit; the query lexical unit sequence is the initial prompt word input for the large language model; The nearest neighbor generation module is used to determine the query semantic nearest neighbor set and query condition probability corresponding to each second query term based on the prefix of the second query term in the query term sequence; The lexical matching module is used to match the query lexical sequence with each stored lexical sequence indicated by the target multi-branch tree based on the text content of the first query lexical, the query semantic nearest neighbor set of the second query lexical, and the query conditional probability, to obtain a matching result; in the target multi-branch tree, the root node is empty, and each other node except the root node is used to store a corresponding stored lexical and the key value vector position information of the stored lexical. The stored lexicals from the root node to any other node constitute a stored lexical sequence, and the common stored lexical of different stored lexical sequences is stored once in the target multi-branch tree; The key value acquisition module is used to acquire the target key value vector of each target stored word based on the key value vector position information of each target stored word in the target stored word sequence when the matching result indicates that there is a target stored word sequence that matches the query word sequence; the target key value vector is used to generate the target prompt word input of the large language model.

[0013] According to a third aspect of this application, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the data query method described in the first aspect above.

[0014] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the data query method described in the first aspect above.

[0015] The data query method, apparatus, device, and medium provided in this application divide the query term sequence into a first query term and a second query term. For each second query term, the corresponding query semantic nearest neighbor set and query condition probability are determined based on its prefix in the query term sequence. Combined with the text content of the first query term, the query term sequence is matched with each stored term sequence indicated by the target multi-way tree. This significantly improves the hit rate and reuse rate of cached key-value vectors, effectively reuses key-value vectors of terms with the same or similar semantics but different expressions, and significantly reduces memory usage while ensuring generation quality and cache hit rate. This improves the overall efficiency and scalability of large language model inference, and achieves synergistic optimization of large language models in three dimensions: memory efficiency, throughput performance, and generation quality. It effectively reduces the problems of low cache hit rate, serious memory redundancy, and unstable generation quality in related technologies.

[0016] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0017] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a data query method in an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating another data query method in an exemplary embodiment of this application; Figure 3 This is a schematic diagram illustrating a data query process according to an exemplary embodiment of this application; Figure 4 This is a schematic diagram of a data query device shown in an exemplary embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device shown in an exemplary embodiment of this application. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0020] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0022] Research has revealed that with the widespread application of large language models in dialogue systems, retrieval augmented generation (RAG), and intelligent customer service, the memory overhead of KV cache has become a major bottleneck in inference deployment. Especially in multi-turn dialogues and scenarios with diverse user input, requests with similar semantics but different literal meanings generate a large number of highly redundant cache paths, leading to low memory utilization and limited cache hit rates.

[0023] To improve cache reuse, existing inference engines, such as the vectorized large language model (vLLM) and the TensorRT-LLM library for large language models, generally adopt exact prefix sharing. They reuse key-value vectors only when the lexical sequences of multiple requests are exactly the same. However, this method has obvious limitations. It only supports exact literal matching and cannot recognize semantically equivalent but differently expressed inputs. This causes a large number of reusable requests to be treated as new requests, resulting in a low cache hit rate. In addition, requests with similar semantics will generate highly similar but independent key-value caches, resulting in a large number of duplicate key-value vectors being stored, which is a serious waste of memory and limits the system's concurrency capabilities.

[0024] Based on the above research, this application provides a data query method, apparatus, device, and medium. By dividing the query term sequence into a first query term and a second query term, and determining the corresponding semantic nearest neighbor set and query condition probability for each second query term based on its prefix in the query term sequence, and combining the text content of the first query term, the query term sequence is matched with each stored term sequence indicated by the target multi-way tree. This significantly improves the hit rate and reuse rate of cached key-value vectors, effectively reuses key-value vectors of terms with the same or similar semantics but different expressions, and significantly reduces memory usage while ensuring generation quality and cache hit rate.

[0025] To facilitate understanding of this embodiment, a data query method disclosed in this application will first be described in detail. The execution entity of the data query method provided in this application is generally an electronic device with a certain computing power. This electronic device can be a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data, and artificial intelligence platforms. In some possible implementations, this data query method can be implemented by a processor calling computer-readable instructions stored in memory.

[0026] The following description, in conjunction with the accompanying drawings, illustrates a data query method provided by an embodiment of this application.

[0027] See Figure 1 The diagram shown is a flowchart illustrating a data query method according to an exemplary embodiment of this application. Figure 1 As shown in the figure, the data query method provided in this embodiment includes steps S101 to S104, wherein: S101: Based on a first preset number, the multiple query words in the query word sequence are divided into a first query word and a second query word; the first query word is the first preset number of query words in the query word sequence, and the second query word is the other query words in the query word sequence besides the first query word; the query word sequence is the initial prompt word input for the large language model.

[0028] In this step, multiple query words in the query word sequence can be divided into first query words and second query words according to a first preset number. The query words that rank first preset number in the word sequence are determined as first query words, and the other query words that rank after the first query words in the query word sequence are determined as second query words.

[0029] It can be understood that the term "lexicon" refers to a token, which is the smallest unit of text processing.

[0030] The specific value of the first preset quantity can be set according to the data query needs, and no specific limitation is made here.

[0031] For example, taking the query term sequence as "what's a good thing to eat after get off work tonight", the query term sequence includes the query terms "today", "evening", "after get off work", "suitable", "eat", and "what". If the first preset quantity is 2, then the first two terms, "today" and "evening", are the first query terms, and the subsequent terms, "after get off work", "suitable", "eat", and "what", are the second query terms.

[0032] S102: For each second query term, based on the prefix of the second query term in the query term sequence, determine the query semantic nearest neighbor set and query condition probability corresponding to the second query term.

[0033] In this step, for each second query term, the query semantic nearest neighbor set and query condition probability corresponding to the second query term can be determined based on the prefix of the second query term in the query term sequence, that is, the preceding text of the second query term in the query term sequence.

[0034] Traditionally, cached query matching is performed by introducing external semantic vectors. For example, a pre-trained embedding model encodes the prompt as a vector, and Approximate Nearest Neighbor Search (ANNS) is used to retrieve similar caches. For instance, taking Retrieval Enhanced Generation (RAG) fusion cache as an example, semantic retrieval ideas from RAG are introduced into KV Cache management. First, the request prompt is encoded into a dense vector using a pre-trained model such as Sentence-BERT. Then, an Approximate Nearest Neighbor Search (ANNS) library, such as Facebook AI Similarity Search (FAISS), is used to retrieve the semantically most similar historical request from the cache vector library, and its KV Cache is reused. However, the semantic space is inconsistent with the distribution of the large model itself, leading to a mismatch between the matching results and the actual generation requirements, posing a risk of domain shift, which may cause semantic drift or a decrease in generation quality. Furthermore, this approach requires the deployment of independent vector indexing and embedding services, increasing system complexity and inference latency, affecting inference efficiency, and introducing additional dependencies and overhead. Furthermore, vector retrieval typically returns the most similar entire path, but it cannot guarantee epoch-level alignment, makes it difficult to safely reuse KV caches with partial prefixes, and makes it difficult to guarantee the context continuity and positional consistency required by attention mechanisms.

[0035] In this embodiment, the semantic nearest neighbor set is intrinsically generated within the reasoning process of the large language model, eliminating the need to introduce external semantic vectors and ensuring consistency between the semantic space and the generation behavior of the large language model itself. This improves matching quality.

[0036] In some possible implementations, determining the query semantic nearest neighbor set and query condition probability corresponding to the second query term based on its prefix in the query term sequence includes: The prefix of the second query term in the query term sequence is input into the large language model to obtain the conditional probability distribution corresponding to the second query term output by the large language model; the conditional probability distribution is used to indicate the probability value of each preset term in the model vocabulary occurring at the position of the second query term under the condition that the prefix of the query term occurs; the model vocabulary is used to record the preset terms that the large language model can output. Based on the probability value corresponding to each preset word element, the second preset number of preset word elements with the highest probability value are determined from the model vocabulary and identified as the target preset word element. The set of target preset words is determined as the query semantic nearest neighbor set corresponding to the second query word; Based on the probability value corresponding to the target preset word, the query condition probability corresponding to the second query word is determined.

[0037] In this step, the prefix of the second query term in the query term sequence is input into the large language model. The large language model performs a forward computation on the prefix of the second query term in the query term sequence to obtain the conditional probability distribution corresponding to the second query term output by the large language model. For example, the conditional probability distribution can be represented by the following formula (1): (1) in, This represents the conditional probability distribution corresponding to the i-th second query term. This represents the probability value of the preset word occurring at the position of the i-th second query word. This represents the prefix of the i-th second query term in the query term sequence. Represents the model vocabulary.

[0038] Here, the conditional probability distribution satisfies and .

[0039] Based on the probability values ​​corresponding to each of the preset word units, a second preset number of preset word units with the highest probability values ​​are determined from the model vocabulary and designated as target preset word units. The specific value of the second preset number can be set according to data query needs and is not specifically limited here. For example, the second preset number is an integer from 3 to 10.

[0040] The set of target preset words is determined as the query semantic nearest neighbor set corresponding to the second query word. For example, the query semantic nearest neighbor set corresponding to the second query word can be represented by the following formula (2): (2) in, This represents the semantic nearest neighbor set corresponding to the i-th second query term. Indicates the target predefined word group, Represents the model vocabulary. Indicates the second preset quantity. This represents the conditional probability distribution corresponding to the i-th second query term.

[0041] Thus, the semantic nearest neighbor set corresponding to each of the second query terms in the query term sequence can be obtained. For example, the semantic nearest neighbor set corresponding to each of the second query terms in the query term sequence can be represented by the following formula (3): (3) in, This represents the set of semantic nearest neighbors corresponding to each second query term in the query term sequence. This represents the semantic nearest neighbor set corresponding to the first second query term. This represents the semantic nearest neighbor set corresponding to the second query term. This represents the semantic nearest neighbor set corresponding to the nth second query term, where n represents the number of second query terms in the query term sequence.

[0042] The average value of the probability values ​​corresponding to the target preset word is determined as the query condition probability corresponding to the second query word.

[0043] In this way, the semantic nearest neighbor set is generated by the semantic space of the large language model itself, which has the characteristics of being system-inherent and having zero external dependencies. It does not require the introduction of external models or vector databases, which can not only avoid the risk of domain shift, but also reduce system complexity and inference latency, and provide reliable support for the matching of subsequent query word sequences with the stored word sequences indicated by the target multi-way tree.

[0044] S103: Based on the text content of the first query term, the query semantic nearest neighbor set of the second query term, and the query condition probability, the query term sequence is matched with each stored term sequence indicated by the target multi-way tree to obtain a matching result; in the target multi-way tree, the root node is empty, and each other node except the root node is used to store a corresponding stored term and the key value vector position information of the stored term. The stored terms from the root node to any other node constitute a stored term sequence, and the common stored terms of different stored term sequences are stored once in the target multi-way tree.

[0045] Here, the target multi-way tree can be in the form of a radix tree or a prefix tree, etc. Based on the above tree structure, a sharing mechanism is added. The common stored lexical units of different stored lexical sequences are stored once in the target multi-way tree, and the suffixes of the common stored lexical units form different sub-sequence branches, representing semantic or literal differences.

[0046] In the target multi-way tree, the root node is empty, does not correspond to any token, and has a depth of 0. Each node other than the root node (including internal nodes and leaf nodes) is used to store a corresponding token and the key-value vector position information of the token. The tokens from the root node to any of the other nodes constitute a token sequence.

[0047] In some possible implementations, in the target multi-branch tree, each node with a hierarchical interval greater than the first preset number of nodes is also used to store the storage semantic nearest neighbor set and storage condition probability of the corresponding storage lexicon.

[0048] The specific process for determining the storage semantic nearest neighbor set and storage condition probability of the stored term is similar to the process for determining the query semantic nearest neighbor set and query condition probability of the second query term in the aforementioned embodiments. For details, please refer to the aforementioned embodiments, and will not be repeated here.

[0049] In this way, by adding lexical-level dynamic semantic nearest neighbor metadata to the above tree structure, the target multi-branch tree can support fuzzy prefix matching based on semantic approximation relationships.

[0050] Specifically, the data stored in each node other than the root node includes the following fields: The token identifier (token_id) is the ID of the stored token corresponding to the current node in the model vocabulary. The token_id is of type integer (int) and is used to indicate the stored token corresponding to the current node. Depth is the index of the current node in the target multi-way tree. The type of depth is integer (int). The key-value vector position index (kv_block_ids) is a list of integers (List[int]). kv_block_ids represents a list of IDs of physical blocks in the cache KV Cache, organized by layer in the cache, such as [block_id_layer0, block_id_layer1, ...]. The key-value vector offset position (block_offsets) is a list of integers (List[int]). Block_offsets is used to represent the offset position of the storage token corresponding to the current node in each physical block. The child node information (children), of type Dict[int,Node], is used to indicate the memory address information of the storage tokens corresponding to the child nodes of the current node; Stores a semantic nearest neighbor set (dynamic_neighbors), where dynamic_neighbors is of type Optional[Set[int]]. Store the conditional probability (avg_topk_prob), where avg_topk_prob is of type floating-point (float). Access count (access_count), which is of type integer (int), is used to indicate the number of times the stored token corresponding to the current node has been successfully matched.

[0051] The key-value vector position information includes the key-value vector position index and the key-value vector offset position.

[0052] It is understood that for each node whose hierarchical interval with respect to the root node is less than or equal to the first preset number, the values ​​for storing semantic nearest neighbor sets and storing conditional probabilities are empty.

[0053] The process involves matching the query term sequence with each stored term sequence indicated by the target multi-way tree based on the text content of the first query term, the query semantic nearest neighbor set of the second query term, and the query conditional probability, to obtain a matching result, including: According to the hierarchical order of the target multi-branch tree, based on the text content of the first query term and the text content of the first stored term in each stored term sequence indicated by the target multi-branch tree, the first query term is sequentially matched with the first stored term in each stored term sequence to obtain the text matching result; the first stored term is the first preset number of stored terms in the stored term sequence. If the text matching result indicates the existence of a candidate stored word sequence that matches the query word sequence text, for each candidate stored word sequence that matches the text, based on the query semantic nearest neighbor set and the query conditional probability of the second query word, and the storage semantic nearest neighbor set and the storage conditional probability of the second stored word, the second query word is sequentially semantically matched with the second stored word in the candidate stored word sequence until the last second query word in the query word sequence is semantically matched successfully or any second query word fails to be semantically matched, thus obtaining a semantic matching result; the second stored word is any other stored word in the stored word sequence other than the first stored word. The candidate stored word sequence indicated by the semantic matching result is determined as the target stored word sequence that matches the query word sequence; the sequence length of the target stored word sequence is less than or equal to the sequence length of the query word sequence.

[0054] In the above steps, each query word in the query word sequence can be traversed position by position, and each query word can be matched with each stored word in each stored word sequence. Only after the current query word is successfully matched will the matching of the next query word be performed, until the last query word in the query word sequence is successfully matched or any query word fails to match.

[0055] The matching performed for each of the first query terms is a text matching. If the text content is consistent, the first query term is considered to have matched successfully; otherwise, the matching fails.

[0056] The matching performed for each of the second query terms is a semantic matching. In some possible implementations, the successful semantic matching of any second query term is determined by the following steps: When the second query term belongs to the storage semantic nearest neighbor set of the second stored term, or when the second stored term belongs to the query semantic nearest neighbor set of the second query term, the query condition probability and the storage condition probability are compared with the probability threshold respectively. If both the query condition probability and the storage condition probability are greater than or equal to the probability threshold, the second query term semantic match is determined to be successful.

[0057] The specific value of the probability threshold can be set according to the data query needs, and no specific limitation is made here.

[0058] In this way, by determining whether the second query term belongs to the storage semantic nearest neighbor set of the second stored term, or whether the second stored term belongs to the query semantic nearest neighbor set of the second query term, and combining the dual verification conditions that both the query condition probability and the storage condition probability are not lower than the probability threshold, the comprehensiveness of semantic matching is improved, the matching error caused by low probability semantic association is effectively avoided, the reliability of semantic matching results is guaranteed, and the accuracy of cached key-value vector reuse is further improved, providing strong support for the stable optimization of large language model inference.

[0059] The candidate stored word sequence indicated by the semantic matching result is determined as the target stored word sequence that matches the query word sequence. This means that the target stored word sequence is the longest stored sequence in the target multi-way tree that matches the query word sequence. If the last query word in the query word sequence matches successfully, the sequence length Lm of the target stored word sequence is equal to the sequence length L of the query word sequence; if any query word fails to match, the sequence length Lm of the target stored word sequence is less than the sequence length L of the query word sequence.

[0060] In this way, precise text matching is performed on the first query term, and fuzzy semantic matching is performed on the second query term. This approach ensures the accuracy of basic matching by relying on text matching, while effectively filtering out stored terms that are semantically similar but have different expressions by using semantic matching. This approach balances cache hit rate with matching efficiency.

[0061] Optionally, after determining the target stored word sequence, the number of successful matches for each target stored word in the target stored word sequence can be incremented by one to update the number of successful matches.

[0062] In some possible implementations, the method further includes: If the sequence length of the target stored word sequence is less than the sequence length of the query word sequence, extract the unmatched query word sequence relative to the target stored word sequence from the query word sequence; The node corresponding to the last stored word in the target stored word sequence is determined as the insertion node; In the target multi-way tree, after the insertion node, corresponding nodes are created for each query term in the sequence of unmatched query terms.

[0063] In the above steps, if the sequence length of the target stored word sequence is less than the sequence length of the query word sequence, and the query words in the query word sequence are not completely covered by the target multi-way tree, then unmatched query word sequences relative to the target stored word sequence can be extracted from the query word sequence and added to the target multi-way tree for future matching.

[0064] For example, the query term sequence is The query term sequence has a sequence length of L, and the target stored term sequence has a sequence length of L. , The unmatched query term sequence is The length of the unmatched query term sequence is .

[0065] The node corresponding to the last stored word in the target stored word sequence is determined as the insertion node. If the sequence length of the target stored word sequence is... If the value is 0, the root node is determined as the insertion node.

[0066] In the target multi-way tree, after the insertion node, corresponding nodes are created for each query term in the unmatched query term sequence, that is, each child node of the insertion node is created sequentially.

[0067] In this way, unmatched parts of the query term sequence are directly inserted into the target multi-way tree. This operation is simple, avoids complex aggregation judgments, and helps reduce inference latency. Furthermore, because the insertion operation is simple, it is compatible with existing tree structures and can be smoothly integrated into inference engines such as vLLM and TensorRT-LLM without modifying the model structure or attention calculation logic. This makes it engineering-friendly and easy to integrate.

[0068] S104: When the matching result indicates the existence of a target stored word sequence that matches the query word sequence, the target key value vector of each target stored word is obtained based on the key value vector position information of each target stored word in the target stored word sequence; the target key value vector is used to generate the target prompt word input of the large language model.

[0069] In this step, for the target stored word sequence that matches the query word sequence, the target key value vector of each target stored word in the target stored word sequence can be obtained. Subsequently, the target key value vector can be reused to generate the target prompt word input of the large language model.

[0070] When the sequence length of the target stored word sequence is less than the sequence length of the query word sequence, for the unmatched query word sequences in the query word sequence relative to the target stored word sequence, each query word in the unmatched query word sequence can be used to generate the target prompt word input of the large language model.

[0071] Traditional methods, such as precise prefix sharing or the introduction of external semantic vectors, lack an active aggregation mechanism. These methods only passively reuse the cached key-value vectors of matched lexical units and do not have the ability to cluster or compress the semantics of cached lexical units, thus failing to reduce the number of redundant caches at the source.

[0072] There is a lack of effective aggregation and compression mechanisms to address the issue of cache redundancy. After a match is found, independent cache key-value vectors may be assigned to semantically similar lexical sequences, leading to memory bloat. If lexical sequences are simply merged when writing to the tree structure, low-frequency, non-representative paths written early may occupy cache positions, suppressing the caching efficiency of high-frequency standard expressions. If pure background compression is relied upon, incorrect clustering (such as merging "recommend horror movies" with "recommend romance movies") may lead to a deterioration in generation quality.

[0073] At the cache management level, eviction policies such as Least Recently Used (LRU) are typically used to release cold caches, or similar paths are simply merged in the background. For example, some inference engines, under memory pressure, will traverse the index and merge the cache key-value vectors of lexical sequences with edit distances less than a threshold. However, the above methods lack verification of semantic compatibility, judging similarity only based on lexical literals or fixed rules, and cannot identify deep semantic differences (such as "recommend horror movies" and "recommend romance movies"). Forced merging will lead to the use of incorrect cache key-value vectors to generate unreasonable model output. In addition, the merging process usually does not preserve the original topology structure, but deletes or modifies the original topology structure, causing subsequent identical requests to miss the cache, thus reducing the cache hit rate. Furthermore, the above methods are mostly static and passively triggered. The compression process does not consider the actual access frequency of the stored word sequence. It may retain low-frequency and non-representative stored word sequences as representatives, while high-frequency stored word sequences are aggregated into aliases, affecting the allocation effect of cache resources. Consequently, it cannot guide the cache to converge towards highly representative word sequences, making it difficult to achieve coordinated optimization of memory efficiency and model service quality.

[0074] In this embodiment of the disclosure, when merging the stored word sequences indicated by the target multi-way tree, other stored word sequences are guided to converge toward the representative stored word sequence with the highest number of successful matches, thereby achieving synergistic optimization of memory efficiency and model service quality. Furthermore, the topology of the target multi-way tree is not modified, so it does not affect the subsequent matching effect and helps to improve the quality of matching results and cache hit rate.

[0075] Please see Figure 2 This is a flowchart illustrating another data query method as shown in an exemplary embodiment of this application. Figure 2 As shown in the figure, the data query method provided in this embodiment includes steps S201 to S209, wherein: S201: Based on a first preset number, the multiple query words in the query word sequence are divided into a first query word and a second query word; the first query word is the first preset number of query words in the query word sequence, and the second query word is the other query words in the query word sequence besides the first query word; the query word sequence is the initial prompt word input of the large language model.

[0076] S202: For each second query term, based on the prefix of the second query term in the query term sequence, determine the query semantic nearest neighbor set and query condition probability corresponding to the second query term.

[0077] S203: Based on the text content of the first query term, the query semantic nearest neighbor set of the second query term, and the query condition probability, the query term sequence is matched with each stored term sequence indicated by the target multi-way tree to obtain a matching result; in the target multi-way tree, the root node is empty, and each other node except the root node is used to store a corresponding stored term and the key value vector position information of the stored term. The stored terms from the root node to any other node constitute a stored term sequence, and the common stored terms of different stored term sequences are stored once in the target multi-way tree.

[0078] S204: When the matching result indicates the existence of a target stored word sequence that matches the query word sequence, the target key value vector of each target stored word is obtained based on the key value vector position information of each target stored word in the target stored word sequence; the target key value vector is used to generate the target prompt word input of the large language model.

[0079] The descriptions of steps S201 to S204 can refer to the descriptions of steps S101 to S104, and can achieve the same technical effect and solve the same technical problem, so they will not be repeated here.

[0080] S205: Construct an undirected graph corresponding to the target multi-way tree; the undirected graph includes a sequence set and an edge set, the sequence set includes the stored word sequence indicated by the target multi-way tree, and the edge set includes the edge formed by connecting two stored word sequences with semantic similarity greater than or equal to a similarity threshold.

[0081] In the aforementioned embodiments, the unmatched query term sequences relative to the target stored term sequence in the query term sequence are added to the target multi-way tree. In this embodiment of the disclosure, periodic merging can be performed to retain a cache key-value vector of more representative stored term sequences, thereby reducing the amount of cached data while ensuring the cache hit rate.

[0082] In this step, all stored word sequences indicated by the target multi-branch tree can be used to form a sequence set. All stored word sequences indicated by the target multi-branch tree can be divided into pairs. The semantic similarity of each stored word sequence pair can be determined. Stored word sequence pairs with semantic similarity greater than or equal to the similarity threshold can be selected from all stored word sequence pairs. The two stored word sequences included in the selected stored word sequence pairs can be connected to form an edge, thereby obtaining the edge set.

[0083] The specific value of the similarity threshold can be set according to the data query needs, and no specific limitation is made here.

[0084] For example, the undirected graph can be represented by the following formula (4): (4) Where G represents an undirected graph, P represents a sequence set, and E represents an edge set.

[0085] The edge set can be represented by the following formula (5): (5) Where E represents the edge set. and Represents two sequences of stored terms. express and Semantic similarity between them This represents the similarity threshold.

[0086] In some possible implementations, the semantic similarity between two stored lexical sequences is determined by the following steps: Determine the sequence length difference, maximum sequence length, and minimum sequence length between two stored word sequences; Based on the maximum sequence length and length tolerance, determine the length difference threshold; If the sequence length difference is less than or equal to the length difference threshold, the semantic similarity between two stored word sequences is determined based on the stored semantic nearest neighbor set of stored words in each stored word sequence and the minimum sequence length. If the sequence length difference is greater than the length difference threshold, the semantic similarity between the two stored word sequences is determined to be 0.

[0087] In the above steps, the sequence length difference, maximum sequence length, and minimum sequence length between the two stored word sequences can be determined based on their respective sequence lengths. The product of the maximum sequence length and the length tolerance is determined as the length difference threshold. The sequence length difference and the length difference threshold are then compared.

[0088] The specific value of the length tolerance can be set according to the data query needs, and is not specifically limited here. For example, the length tolerance can be 0.2.

[0089] If the sequence length difference is less than or equal to the length difference threshold, for each position under the minimum sequence length, determine the number of intersection words included in the intersection of the semantic nearest neighbor sets of the two stored words corresponding to the same position in the two stored word sequences, sum the number of intersection words corresponding to each position under the minimum sequence length, divide the summation result by the minimum sequence length and the second preset number, and use the result as the semantic similarity between the two stored word sequences.

[0090] If the sequence length difference is greater than the length difference threshold, it indicates that the deviation between the two stored word sequences is too large, and the semantic similarity between the two stored word sequences can be directly determined to be 0.

[0091] For example, the semantic similarity between two stored word sequences can be represented by the following formula (6): (6) in, and Represents two sequences of stored terms. express and Semantic similarity between them express and Difference in sequence length between them Indicates length tolerance. express and The maximum sequence length between them, L represents the length difference threshold. and Minimum sequence length between express In position The storage semantic nearest neighbor set of the stored words, express In position The storage semantic nearest neighbor set of the stored words, This indicates the second preset quantity.

[0092] In this way, a dual semantic similarity determination is constructed, which combines length verification and comparison of stored semantic nearest neighbors. This not only avoids invalid semantic calculations caused by excessive differences in sequence lengths, thus improving the efficiency of semantic similarity determination, but also ensures the accuracy of similarity calculation results by relying on stored semantic nearest neighbors, reducing the possibility of erroneous merging. This provides accurate and reliable determination criteria for subsequent undirected graph construction, connected component clustering, and KV cache sharing, thereby helping to reduce the memory footprint of large language model inference systems and improve the overall performance of the system.

[0093] S206: Perform a depth-first search on the undirected graph to extract multiple connected components; the connected components include multiple semantically similar stored word sequences.

[0094] In this step, the depth-first search (DFS) algorithm can be used to extract multiple connected components from the undirected graph.

[0095] For example, the connected components of an undirected graph can be represented by the following formula (7): (7) in, Represents the connected components of an undirected graph G. Indicates the first connected component. Indicates the second connected component. This represents the m-th connected component.

[0096] It is understandable that since undirected graphs are based on the connection between two stored word sequences with semantic similarity greater than or equal to a similarity threshold, each connected component is theoretically composed of multiple stored word sequences representing similar semantics.

[0097] S207: For each of the connected components, determine the representative stored word sequence with the highest number of successful matches from the plurality of stored word sequences included in the connected component.

[0098] In this step, for each connected component, the number of successful matches of each stored word sequence included in the connected component is determined, and the stored word sequence with the highest number of successful matches is determined as the representative stored word sequence corresponding to the connected component.

[0099] Specifically, for each of the stored word sequences, the average number of successful matches of all stored words included in the stored word sequence is determined as the number of successful matches of the stored word sequence.

[0100] For example, the number of successful matches of a stored word sequence can be represented by the following formula (8): (8) in, L represents the number of successful matches of the stored word sequence P, and L represents the number of stored words included in the stored word sequence P. This represents the number of successful matches for the i-th stored word in the stored word sequence P.

[0101] The representative storage word sequence corresponding to the connected components can be represented by the following formula (9): (9) in, This represents a connected component. The stored word sequence, Represents connected components The number of successful matches of the word sequence P is stored in the .

[0102] In this way, the storage word sequence with the highest number of successful matches is determined as the representative storage word sequence corresponding to the connected component, so that the high-frequency storage word sequence serves as the representative of the connected component, thereby improving the allocation effect of cache resources.

[0103] S208: For each other stored word sequence in the connected component other than the representative stored word sequence, match the other stored word sequence with the representative stored word sequence.

[0104] In this step, each other stored word sequence in the connected component, except for the representative stored word sequence, is matched with the representative stored word sequence, thereby guiding the other stored word sequences in the connected component to converge towards the representative stored word sequence, achieving synergistic optimization of memory efficiency and model service quality.

[0105] Here, the specific steps for matching the representative stored word sequence with the other stored word sequences are similar to the steps for matching the query word sequence with each stored word sequence indicated by the target multi-way tree in the previous embodiment. For details, please refer to the previous embodiment, and they will not be repeated here.

[0106] After matching the other stored word sequences with the representative stored word sequence, the longest matching sequence length between the other stored word sequences and the representative stored word sequence can be obtained. The longest matching sequence length is less than or equal to the sequence length of the other stored word sequences.

[0107] S209: For the successfully matched stored lexical units in the other stored lexical unit sequences, adjust the key-value vector position information of the node corresponding to the matched stored lexical unit to the key-value vector position information of the corresponding matched stored lexical unit in the representative stored lexical unit sequence, and delete the key-value vector of the matched stored lexical unit.

[0108] In this step, the stored lexical units in the other stored lexical sequences that correspond to the length of the longest matching sequence are identified as successfully matched stored lexical units. For each matched stored lexical unit, the key-value vector position information stored in the node corresponding to the matched stored lexical unit is adjusted to the key-value vector position information of the corresponding matched stored lexical unit in the representative stored lexical sequence. Specifically, the key-value vector position information stored in the node corresponding to the matched stored lexical unit can be redirected to the key-value vector position information of the corresponding matched stored lexical unit in the representative stored lexical sequence, and the key-value vector of the matched stored lexical unit is deleted to reduce memory usage and overhead. The key-value vector position information stored in the node corresponding to the unmatched stored lexical units in the other stored lexical sequences, as well as the key-value vectors of the stored lexical units themselves, are retained. In this way, adjusting the key-value vector position information stored in the node corresponding to the matched stored lexical unit to the key-value vector position information of the corresponding matched stored lexical unit in the representative stored lexical sequence achieves the merging of semantically compatible stored lexical sequences, significantly reducing redundant storage.

[0109] Here, the topological structure of the other stored term sequences in the target multi-way tree is preserved without deletion or modification. This way, when a new query term sequence is obtained for matching in the future, the stored term sequence can still be used for matching, which helps to improve the quality of matching results and cache hit rate.

[0110] Optionally, the merging of stored word sequences can be performed asynchronously, for example, by aggregating at preset time intervals, such as merging stored word sequences every 24 hours, without affecting the matching of query word sequences and cache calls.

[0111] This implementation clusters semantically similar stored word sequences. For successfully matched stored words, it retains only the key-value cache of the corresponding matched stored words in the representative stored word sequence included in the connected component. The matched stored words share the key-value cache of the corresponding matched stored words in the representative stored word sequence, which greatly reduces the memory usage of the key-value cache, significantly improves the single-machine concurrency capability and deployment density, and further enhances the memory utilization efficiency and scalability of the large model inference system.

[0112] For a clearer illustration of the data query process, see [link to relevant documentation]. Figure 3 This is a schematic diagram illustrating a data query process as shown in an exemplary embodiment of this application. Figure 3 As shown, based on a first preset number, the query term sequence is divided into query term segments to obtain a first query term and a second query term. The first query term is the first preset number of query terms in the query term sequence, and the second query term is the other query terms in the query term sequence besides the first query term. For each second query term, semantic nearest neighbors are generated based on the prefix of the second query term in the query term sequence to determine the query semantic nearest neighbor set and query condition probability corresponding to the second query term. Based on the text content of the first query term, the query semantic nearest neighbor set of the second query term, and the query condition probability... The query term sequence is matched with each stored term sequence indicated by the target multi-way tree to obtain a matching result. If the matching result indicates the existence of a target stored term sequence that matches the query term sequence, the target key-value vector of each target stored term is obtained based on the key-value vector position information of each target stored term in the target stored term sequence. If the sequence length of the target stored term sequence is less than the sequence length of the query term sequence, the unmatched query term sequences relative to the target stored term sequence are inserted into the target multi-way tree. The stored term sequences in the target multi-way tree are then merged. Specific steps are described in the foregoing embodiments and will not be repeated here.

[0113] The data query method provided in this application divides the query term sequence into a first query term and a second query term. For each second query term, the corresponding semantic nearest neighbor set and query condition probability are determined based on its prefix in the query term sequence. Combined with the text content of the first query term, the query term sequence is matched with each stored term sequence indicated by the target multi-way tree. This significantly improves the hit rate and reuse rate of cached key-value vectors, effectively reuses key-value vectors of terms with the same or similar semantics but different expressions. While ensuring generation quality and cache hit rate, it significantly reduces memory consumption, improves the overall efficiency and scalability of large language model inference, and achieves synergistic optimization of large language models in three dimensions: memory efficiency, throughput performance, and generation quality. This effectively reduces the problems of low cache hit rate, serious memory redundancy, and unstable generation quality in related technologies.

[0114] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0115] Corresponding to the aforementioned embodiments of the data query method, this application also provides embodiments of the data query device.

[0116] Please see Figure 4 This is a schematic diagram illustrating a data query device according to an exemplary embodiment of this application. Figure 4 As shown in the figure, the data query device 400 provided in this application embodiment includes: The lexical classification module 401 is used to divide multiple query lexical units in the query lexical unit sequence into a first query lexical unit and a second query lexical unit based on a first preset number; the first query lexical unit is the first preset number of query lexical units in the query lexical unit sequence, and the second query lexical unit is the other query lexical units in the query lexical unit sequence besides the first query lexical unit; the query lexical unit sequence is the initial prompt word input for the large language model; The nearest neighbor generation module 402 is used to determine the query semantic nearest neighbor set and query condition probability corresponding to each second query term based on the prefix of the second query term in the query term sequence; The lexical matching module 403 is used to match the query lexical sequence with each stored lexical sequence indicated by the target multi-way tree based on the text content of the first query lexical, the query semantic nearest neighbor set of the second query lexical, and the query conditional probability, to obtain a matching result; in the target multi-way tree, the root node is empty, and each other node except the root node is used to store a corresponding stored lexical and the key value vector position information of the stored lexical. The stored lexicals from the root node to any other node constitute a stored lexical sequence, and the common stored lexical of different stored lexical sequences is stored once in the target multi-way tree; The key value acquisition module 404 is used to acquire the target key value vector of each target stored word based on the key value vector position information of each target stored word in the target stored word sequence when the matching result indicates that there is a target stored word sequence that matches the query word sequence; the target key value vector is used to generate the target prompt word input of the large language model.

[0117] In some possible implementations, when the nearest neighbor generation module 402 determines the query semantic nearest neighbor set and query condition probability corresponding to the second query term based on the prefix of the second query term in the query term sequence, it is specifically used for: The prefix of the second query term in the query term sequence is input into the large language model to obtain the conditional probability distribution corresponding to the second query term output by the large language model; the conditional probability distribution is used to indicate the probability value of each preset term in the model vocabulary occurring at the position of the second query term under the condition that the prefix of the query term occurs; the model vocabulary is used to record the preset terms that the large language model can output. Based on the probability value corresponding to each preset word element, the second preset number of preset word elements with the highest probability value are determined from the model vocabulary and identified as the target preset word element. The set of target preset words is determined as the query semantic nearest neighbor set corresponding to the second query word; Based on the probability value corresponding to the target preset word, the query condition probability corresponding to the second query word is determined.

[0118] In some possible implementations, in the target multi-way tree, each node with a hierarchical interval greater than the first preset number of nodes is also used to store the storage semantic nearest neighbor set and storage condition probability of the corresponding storage word. The word matching module 403 is specifically used for: According to the hierarchical order of the target multi-branch tree, based on the text content of the first query term and the text content of the first stored term in each stored term sequence indicated by the target multi-branch tree, the first query term is sequentially matched with the first stored term in each stored term sequence to obtain the text matching result; the first stored term is the first preset number of stored terms in the stored term sequence. If the text matching result indicates the existence of a candidate stored word sequence that matches the query word sequence text, for each candidate stored word sequence that matches the text, based on the query semantic nearest neighbor set and the query conditional probability of the second query word, and the storage semantic nearest neighbor set and the storage conditional probability of the second stored word, the second query word is sequentially semantically matched with the second stored word in the candidate stored word sequence until the last second query word in the query word sequence is semantically matched successfully or any second query word fails to be semantically matched, thus obtaining a semantic matching result; the second stored word is any other stored word in the stored word sequence other than the first stored word. The candidate stored word sequence indicated by the semantic matching result is determined as the target stored word sequence that matches the query word sequence; the sequence length of the target stored word sequence is less than or equal to the sequence length of the query word sequence.

[0119] In some possible implementations, the lexical matching module 403 determines that any of the second query lexical terms has successfully matched semantically through the following steps: When the second query term belongs to the storage semantic nearest neighbor set of the second stored term, or when the second stored term belongs to the query semantic nearest neighbor set of the second query term, the query condition probability and the storage condition probability are compared with the probability threshold respectively. If both the query condition probability and the storage condition probability are greater than or equal to the probability threshold, the second query term semantic match is determined to be successful.

[0120] In some possible implementations, the lexical matching module 403 is further configured to: If the sequence length of the target stored word sequence is less than the sequence length of the query word sequence, extract the unmatched query word sequence relative to the target stored word sequence from the query word sequence; The node corresponding to the last stored word in the target stored word sequence is determined as the insertion node; In the target multi-way tree, after the insertion node, corresponding nodes are created for each query term in the sequence of unmatched query terms.

[0121] In some possible implementations, the data query device 400 further includes a storage merging module 405, which is specifically used for: Construct an undirected graph corresponding to the target multi-way tree; the undirected graph includes a sequence set and an edge set, the sequence set includes the stored word sequence indicated by the target multi-way tree, and the edge set includes the edge formed by connecting two stored word sequences with semantic similarity greater than or equal to a similarity threshold; A depth-first search is performed on the undirected graph to extract multiple connected components; each connected component includes multiple semantically similar stored word sequences. For each connected component, the representative stored word sequence with the highest number of successful matches is determined from the multiple stored word sequences included in the connected component; For each other stored word sequence in the connected component other than the representative stored word sequence, match the other stored word sequence with the representative stored word sequence; For the successfully matched stored lexical units in the other stored lexical unit sequences, the key-value vector position information of the node corresponding to the matched stored lexical unit is adjusted to the key-value vector position information of the corresponding matched stored lexical unit in the representative stored lexical unit sequence, and the key-value vector of the matched stored lexical unit is deleted.

[0122] In some possible implementations, the storage merging module 405 determines the semantic similarity between two stored lexical sequences through the following steps: Determine the sequence length difference, maximum sequence length, and minimum sequence length between two stored word sequences; Based on the maximum sequence length and length tolerance, determine the length difference threshold; If the sequence length difference is less than or equal to the length difference threshold, the semantic similarity between two stored word sequences is determined based on the stored semantic nearest neighbor set of stored words in each stored word sequence and the minimum sequence length. If the sequence length difference is greater than the length difference threshold, the semantic similarity between the two stored word sequences is determined to be 0.

[0123] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0124] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0125] Based on the same technical concept, this application also provides a computer device 500, referring to... Figure 5 The diagram shown is a schematic representation of the structure of a computer device according to an exemplary embodiment of this application, comprising: The processor 510, memory 520, and bus 530 are included. The memory 520 is used to store execution instructions and includes main memory 521 and external memory 522. The main memory 521, also known as internal memory, is used to temporarily store the operation data in the processor 510 and the data exchanged with external memory 522 such as hard disk. The processor 510 exchanges data with external memory 522 through main memory 521.

[0126] In this embodiment, the memory 520 is specifically used to store application code that executes the solution of this application, and its execution is controlled by the processor 510. That is, when the electronic device 500 is running, the processor 510 communicates with the memory 520 through the bus 530, or the processor 510 communicates with the memory 520 through other means, so that the processor 510 executes the application code stored in the memory 520, and then executes the steps of the data query method described in any of the foregoing embodiments.

[0127] The memory 520 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0128] Processor 510 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0129] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 500. In other embodiments of this application, the electronic device 500 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0130] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the data query method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0131] This disclosure also provides a computer program product, which stores a computer program. When the computer program is run by a processor, it executes the steps of the data query method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here.

[0132] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0133] Furthermore, embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0134] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0135] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0136] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0137] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0138] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0139] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0140] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data query method, characterized in that, The method includes: Based on a first preset number, multiple query words in the query word sequence are divided into a first query word and a second query word; the first query word is the first preset number of query words in the query word sequence, and the second query word is the other query words in the query word sequence besides the first query word; the query word sequence is the initial prompt word input for the large language model; For each second query term, based on the prefix of the second query term in the query term sequence, determine the query semantic nearest neighbor set and query condition probability corresponding to the second query term; Based on the text content of the first query term, the query semantic nearest neighbor set of the second query term, and the query condition probability, the query term sequence is matched with each stored term sequence indicated by the target multi-way tree to obtain a matching result; in the target multi-way tree, the root node is empty, and each other node except the root node is used to store a corresponding stored term and the key value vector position information of the stored term. The stored terms from the root node to any other node constitute a stored term sequence, and the common stored terms of different stored term sequences are stored once in the target multi-way tree; When the matching result indicates the existence of a target stored word sequence that matches the query word sequence, the target key value vector of each target stored word is obtained based on the key value vector position information of each target stored word in the target stored word sequence; the target key value vector is used to generate the target prompt word input of the large language model.

2. The method according to claim 1, characterized in that, The step of determining the semantic nearest neighbor set and query condition probability corresponding to the second query term based on its prefix in the query term sequence includes: The prefix of the second query term in the query term sequence is input into the large language model to obtain the conditional probability distribution corresponding to the second query term output by the large language model; the conditional probability distribution is used to indicate the probability value of each preset term in the model vocabulary occurring at the position of the second query term under the condition that the prefix of the query term occurs; the model vocabulary is used to record the preset terms that the large language model can output. Based on the probability value corresponding to each preset word element, the second preset number of preset word elements with the highest probability value are determined from the model vocabulary and identified as the target preset word element. The set of target preset words is determined as the query semantic nearest neighbor set corresponding to the second query word; Based on the probability value corresponding to the target preset word, the query condition probability corresponding to the second query word is determined.

3. The method according to claim 1, characterized in that, In the target multi-way tree, each node with a hierarchical interval greater than the first preset number of nodes is also used to store the storage semantic nearest neighbor set and storage condition probability of the corresponding storage word. The process involves matching the query term sequence with each stored term sequence indicated by the target multi-way tree based on the text content of the first query term, the query semantic nearest neighbor set of the second query term, and the query conditional probability, to obtain a matching result, including: According to the hierarchical order of the target multi-branch tree, based on the text content of the first query term and the text content of the first stored term in each stored term sequence indicated by the target multi-branch tree, the first query term is sequentially matched with the first stored term in each stored term sequence to obtain the text matching result; the first stored term is the first preset number of stored terms in the stored term sequence. If the text matching result indicates the existence of a candidate stored word sequence that matches the query word sequence text, for each candidate stored word sequence that matches the text, based on the query semantic nearest neighbor set and the query conditional probability of the second query word, and the storage semantic nearest neighbor set and the storage conditional probability of the second stored word, the second query word is sequentially semantically matched with the second stored word in the candidate stored word sequence until the last second query word in the query word sequence is semantically matched successfully or any second query word fails to be semantically matched, thus obtaining a semantic matching result; the second stored word is any other stored word in the stored word sequence other than the first stored word. The candidate stored word sequence indicated by the semantic matching result is determined as the target stored word sequence that matches the query word sequence; the sequence length of the target stored word sequence is less than or equal to the sequence length of the query word sequence.

4. The method according to claim 3, characterized in that, The following steps are used to determine if any of the second query terms has successfully matched semantically: When the second query term belongs to the storage semantic nearest neighbor set of the second stored term, or when the second stored term belongs to the query semantic nearest neighbor set of the second query term, the query condition probability and the storage condition probability are compared with the probability threshold respectively. If both the query condition probability and the storage condition probability are greater than or equal to the probability threshold, the second query term semantic match is determined to be successful.

5. The method according to claim 3, characterized in that, The method further includes: If the sequence length of the target stored word sequence is less than the sequence length of the query word sequence, extract the unmatched query word sequence relative to the target stored word sequence from the query word sequence; The node corresponding to the last stored word in the target stored word sequence is determined as the insertion node; In the target multi-way tree, after the insertion node, corresponding nodes are created for each query term in the sequence of unmatched query terms.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Construct an undirected graph corresponding to the target multi-way tree; the undirected graph includes a sequence set and an edge set, the sequence set includes the stored word sequence indicated by the target multi-way tree, and the edge set includes the edge formed by connecting two stored word sequences with semantic similarity greater than or equal to a similarity threshold; A depth-first search is performed on the undirected graph to extract multiple connected components; each connected component includes multiple semantically similar stored word sequences. For each connected component, the representative stored word sequence with the highest number of successful matches is determined from the multiple stored word sequences included in the connected component; For each other stored word sequence in the connected component other than the representative stored word sequence, match the other stored word sequence with the representative stored word sequence; For the successfully matched stored lexical units in the other stored lexical unit sequences, the key-value vector position information of the node corresponding to the matched stored lexical unit is adjusted to the key-value vector position information of the corresponding matched stored lexical unit in the representative stored lexical unit sequence, and the key-value vector of the matched stored lexical unit is deleted.

7. The method according to claim 6, characterized in that, The semantic similarity between two stored word sequences is determined by the following steps: Determine the sequence length difference, maximum sequence length, and minimum sequence length between two stored word sequences; Based on the maximum sequence length and length tolerance, determine the length difference threshold; If the sequence length difference is less than or equal to the length difference threshold, the semantic similarity between two stored word sequences is determined based on the stored semantic nearest neighbor set of stored words in each stored word sequence and the minimum sequence length. If the sequence length difference is greater than the length difference threshold, the semantic similarity between the two stored word sequences is determined to be 0.

8. A data query device, characterized in that, The device includes: The lexical classification module is used to divide multiple query lexical units in the query lexical unit sequence into a first query lexical unit and a second query lexical unit based on a first preset number; the first query lexical unit is the first preset number of query lexical units in the query lexical unit sequence, and the second query lexical unit is the other query lexical units in the query lexical unit sequence besides the first query lexical unit; the query lexical unit sequence is the initial prompt word input for the large language model; The nearest neighbor generation module is used to determine the query semantic nearest neighbor set and query condition probability corresponding to each second query term based on the prefix of the second query term in the query term sequence; The lexical matching module is used to match the query lexical sequence with each stored lexical sequence indicated by the target multi-branch tree based on the text content of the first query lexical, the query semantic nearest neighbor set of the second query lexical, and the query conditional probability, to obtain a matching result; in the target multi-branch tree, the root node is empty, and each other node except the root node is used to store a corresponding stored lexical and the key value vector position information of the stored lexical. The stored lexicals from the root node to any other node constitute a stored lexical sequence, and the common stored lexical of different stored lexical sequences is stored once in the target multi-branch tree; The key value acquisition module is used to acquire the target key value vector of each target stored word based on the key value vector position information of each target stored word in the target stored word sequence when the matching result indicates that there is a target stored word sequence that matches the query word sequence; the target key value vector is used to generate the target prompt word input of the large language model.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the data query method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the data query method according to any one of claims 1 to 7.

Images