Information retrieval method, apparatus and electronic device

Abstract

The application discloses an information retrieval method and device and electronic equipment. The method receives a query request, the query request representing a request for information retrieval on text content in a knowledge base; obtains target cache data matched with the query request from a target cache library, wherein the target cache library is constructed in advance based on the text content in the knowledge base and is used for storing key-value caches of at least one semantic unit corresponding to the text content in the knowledge base, and the semantic unit is obtained by dividing the text content based on semantic correlation degrees between sentences in the text content; and based on the target cache data, outputs a target retrieval result corresponding to the query request.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically to an information retrieval method, apparatus, and electronic device. Background Technology

[0002] With the rapid development of information technology, knowledge base question-and-answer systems are widely used in enterprise internal management, intelligent customer service and other scenarios, allowing users to quickly obtain the information they need based on natural language queries.

[0003] However, traditional information retrieval methods often rely on real-time retrieval operations when processing large-scale knowledge bases, resulting in high response latency and impacting user experience. Furthermore, when converting document content into searchable knowledge units during processing, the methods often fail to effectively maintain the semantic integrity of the original document, easily leading to fragmented information in the search results and affecting the accuracy of subsequent information output. Summary of the Invention

[0004] This application provides the following technical solution:

[0005] An information retrieval method, comprising:

[0006] Receive a query request, which represents a request to retrieve information from text content in a knowledge base;

[0007] Obtain target cache data matching the query request from the target cache library, wherein the target cache library is a key-value cache pre-built based on the text content in the knowledge base, used to store at least one semantic unit corresponding to the text content in the knowledge base, and the semantic unit is obtained by dividing the text content based on the semantic correlation between sentences in the text content;

[0008] Based on the target cached data, output the target retrieval result corresponding to the query request.

[0009] Optionally, the semantic unit is obtained in the following way:

[0010] Sentence-level segmentation is performed on the text content in the knowledge base to obtain a sentence sequence;

[0011] Semantic encoding is performed on adjacent sentences in the sentence sequence to obtain semantic feature vectors corresponding to each sentence;

[0012] Based on the semantic feature vectors of adjacent sentences, the semantic similarity of the adjacent sentences is determined, and the semantic similarity is determined as the semantic relevance of the adjacent sentences;

[0013] Based on the semantic relevance, the semantic unit division boundaries in the sentence sequence are determined;

[0014] The sentence sequence is divided based on the semantic unit partitioning boundary to obtain at least one semantic unit.

[0015] Optionally, determining the semantic unit partitioning boundaries in the sentence sequence based on the semantic relevance includes:

[0016] In response to the semantic correlation between adjacent sentences being less than a preset threshold, or in response to the cumulative length of the current semantic unit reaching a preset length threshold, the current position is determined as the semantic unit division boundary;

[0017] The step of dividing the sentence sequence based on the semantic unit partitioning boundary to obtain the at least one semantic unit includes:

[0018] The sentence sequence is divided based on the semantic unit partitioning boundaries to obtain initial semantic units;

[0019] The adjacent initial semantic units are overlapped so that the latter semantic unit contains the tail information of the former semantic unit.

[0020] Optionally, the key-value cache is obtained in the following way:

[0021] Each semantic unit is vectorized to obtain the embedded representation of each semantic unit;

[0022] Based on the key projection matrix and value projection matrix of the language model, the embedding representation is projected and transformed to generate the key cache matrix and value cache matrix for each semantic unit.

[0023] The key cache matrix and value cache matrix of each semantic unit are concatenated to obtain the global key-value cache corresponding to the text content in the knowledge base;

[0024] Establish an index association between the text content, the semantic unit, and the storage location in the global key-value cache, and store it in the target cache library.

[0025] Optionally, obtaining the target cache data matching the query request from the target cache library includes:

[0026] The query request is vectorized to obtain an embedded representation of the query request;

[0027] The query projection matrix is ​​obtained by projecting the embedded representation of the query request based on the language model.

[0028] Calculate the similarity score between each query element in the query projection matrix and each cache element in the key-value cache of the target cache library;

[0029] Based on the similarity score, the target cached data that matches the query request is determined from the key-value cache.

[0030] Optionally, calculating the similarity score between each query element in the query projection matrix and each cached element in the key-value cache of the target cache library includes:

[0031] Based on the target dimension set, the similarity sub-parameters between each query element in the query projection matrix and each cache element in the key-value cache of the target cache library are calculated respectively; wherein, the target dimension set includes at least a first dimension for characterizing semantic direction consistency, a second dimension for characterizing semantic matching strength, and a third dimension for characterizing semantic detail differences;

[0032] The similarity sub-parameters are fused to obtain the similarity score.

[0033] Optionally, determining the target cached data matching the query request from the key-value cache based on the similarity score includes:

[0034] Based on the distribution characteristics of the similarity scores, a dynamic matching threshold is determined;

[0035] The cached elements whose similarity scores meet the dynamic matching threshold are identified as the target cached data;

[0036] The step of outputting the target retrieval result corresponding to the query request based on the target cache data includes:

[0037] Based on the similarity score, the attention weight corresponding to each target cached data is determined;

[0038] The value cache matrix corresponding to the target cache data is weighted and aggregated based on the attention weights to generate the context representation of the query request;

[0039] The context representation is input into the language model to generate the target retrieval result.

[0040] Optionally, it also includes:

[0041] Based on the frequency of association between historical queries and semantic units, a target query association graph is constructed.

[0042] In response to the query request, determine the historical queries that are semantically associated with the query request from the target query association graph;

[0043] Based on the target semantic unit corresponding to the historical query, the key value of the target semantic unit is cached in the target storage area;

[0044] The step of retrieving target cache data matching the query request from the target cache library includes:

[0045] Retrieve the target cache data that matches the query request from the target storage area.

[0046] An information retrieval device, comprising:

[0047] A receiving unit is used to receive a query request, wherein the query request represents a request to retrieve information from text content in a knowledge base;

[0048] A matching unit is used to retrieve target cache data that matches the query request from the target cache library, wherein the target cache library is pre-built based on the text content in the knowledge base and is used to store key-value caches containing at least one semantic unit corresponding to the text content in the knowledge base, wherein the semantic unit is obtained by dividing the text content based on the semantic correlation between sentences in the text content;

[0049] The output unit is used to output the target retrieval result corresponding to the query request based on the target cached data.

[0050] An electronic device includes at least one processor and a target prediction model capable of running on the processor, the processor being configured to perform the following operations:

[0051] Receive a query request, which represents a request to retrieve information from text content in a knowledge base;

[0052] Obtain target cache data matching the query request from the target cache library, wherein the target cache library is a key-value cache pre-built based on the text content in the knowledge base, used to store at least one semantic unit corresponding to the text content in the knowledge base, and the semantic unit is obtained by dividing the text content based on the semantic correlation between sentences in the text content;

[0053] Based on the target cached data, output the target retrieval result corresponding to the query request. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0055] Figure 1A flowchart illustrating an information retrieval method provided in an embodiment of this application;

[0056] Figure 2 A flowchart illustrating a method for generating semantic units provided in an embodiment of this application;

[0057] Figure 3 This application provides an interactive schematic diagram of an information retrieval system in an application scenario.

[0058] Figure 4 This is a schematic diagram of the structure of an information retrieval device provided in an embodiment of this application. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0060] The terms "first" and "second," etc., used in this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units may include steps or units not listed, but may include steps or units not listed.

[0061] This application provides an information retrieval method applicable to knowledge base question-and-answer scenarios based on large language models, including enterprise internal knowledge bases, intelligent customer service systems, product manual question-and-answer, and legal or technical document retrieval—application environments requiring rapid and accurate information retrieval from large amounts of unstructured text. In this scenario, the processing system of an electronic device receives a query request initiated by a user, specifically a request to retrieve information from text content stored in a knowledge base. Instead of directly performing real-time retrieval in the original knowledge base or vector database as in general processing models, this application retrieves target cache data matching the query request from a pre-built target cache library. This target cache library is pre-built based on the original text content in the knowledge base and internally stores key-value caches of at least one semantic unit corresponding to this text content. These semantic units are not obtained through fixed-length segmentation but are generated by dynamically dividing the original text based on the semantic relationships between sentences. Based on the acquired target cache data, the target retrieval result corresponding to the query request is output. By adopting the above method, the real-time retrieval process in the question-and-answer process is no longer relied upon, thereby reducing the response time of the first character. At the same time, since the cached content is built on semantic integrity, the accuracy and semantic coherence of the retrieval results are also effectively improved.

[0062] See Figure 1 The diagram illustrates a flowchart of an information retrieval method provided in an embodiment of this application. The method may include the following steps:

[0063] S101, Receive query request.

[0064] A query request refers to a natural language question or instruction entered by a user through the interactive interface of a knowledge base question-and-answer system. This instruction can be either text or voice. The query request represents a request to retrieve information from text content within the knowledge base; that is, the query request uses the text content stored in the knowledge base as the retrieval object and target scope. Here, the text content in the knowledge base refers to the raw text data stored in the knowledge base system. In the actual storage format of a Retrieval-Augmented Generation (RAG) knowledge base, the text content is typically preprocessed, broken down into text fragments, and corresponding vector representations are generated, rather than being stored as a complete original document.

[0065] Specifically, the knowledge base question-answering system receives user query requests through a front-end interactive interface. This interface can take the form of a web page dialog box, instant messaging window, voice input interface, or application programming interface (API). Users input questions in natural language, such as "What is the warranty period for a certain product?" or "What is the application process for annual leave at my company?". After receiving the query request, the system parses it to confirm that the search scope is limited to text content already stored in the current knowledge base. For example, in an internal enterprise knowledge base scenario, when the system receives the query request "What is the application process for annual leave at my company?", it determines that the search object is text content related to personnel policies in the enterprise knowledge base.

[0066] S102. Retrieve the target cache data that matches the query request from the target cache library.

[0067] The target cache library is a pre-built key-value cache based on text content in a knowledge base, used to store at least one semantic unit corresponding to the text content in the knowledge base. The semantic unit is obtained by dividing the text content based on the semantic correlation between sentences. Unlike common fixed-length segmentation methods (such as segmenting into blocks of 512 characters or 1024 tokens), the division of semantic units is not solely based on fixed length. Instead, it comprehensively considers the degree of semantic correlation between sentences, grouping semantically closely related sentences into the same unit, thereby ensuring the semantic integrity of each semantic unit.

[0068] Key-value caching refers to the key and value matrices generated after semantic units are processed by a language model. In the language model of the Transformer architecture, the self-attention mechanism requires the computation of three matrices: query, key, and value. The key matrix is ​​used to calculate similarity with the query matrix to determine attention weights, and the value matrix is ​​used to perform a weighted sum based on the attention weights to generate a contextual representation. In this application, key-value caching refers to the pre-computed and stored key and value matrices. In this embodiment, the target cache data refers to the key-value cache data obtained from the target cache library that matches the user's query request.

[0069] The target cache library is pre-built during system initialization or when the knowledge base text content is updated. The text content in the knowledge base can be divided into semantic units to obtain at least one semantic unit. Each semantic unit is processed to generate a corresponding key-value cache, and all key-value caches are stored in the target cache library.

[0070] When the electronic device's processing system receives a user's query request, it no longer performs real-time similarity retrieval in a vector database as in common retrieval enhancement methods. Instead, it directly retrieves target cache data matching the query request from a pre-built target cache library. The matching process is achieved by calculating the semantic similarity between the query request and the key-value cache. During the matching process, all key-value caches in the target cache library are traversed or indexed, the matching degree between each cache element and the query request is calculated, and one or more cache elements with the highest matching degree are identified as the target cache data.

[0071] S103. Based on the target cache data, output the target retrieval results corresponding to the query request.

[0072] The target retrieval result is the answer to the user's query request generated based on the target cached data. For example, the target retrieval result can be a piece of natural language text, and may further include additional information such as citation source and confidence level.

[0073] In one embodiment of this application, target cached data can be processed using a language model to obtain target retrieval results. During the generation process, the value cache matrix in the target cached data is combined with the context representation of the query request, and the answer text is generated word-by-word through the feedforward network and decoding layer of the language model.

[0074] This application provides an information retrieval method that receives a query request for text content in a knowledge base. It retrieves target cache data matching the query request from a pre-built target cache library, which stores key-value caches of semantic units derived from the semantic relationships between sentences. Since the target cache library is pre-built based on the text content in the knowledge base, real-time vector retrieval is unnecessary during queries, thus eliminating reliance on the high-volume real-time retrieval stage in general retrieval enhancement methods and reducing initial response time. Furthermore, because the semantic units corresponding to the key-value caches are dynamically divided based on the semantic relationships between sentences, rather than using a general fixed-length segmentation method, each cache unit maintains the semantic integrity of the original text, resolving semantic fragmentation and information breakage issues caused by segmentation, thereby improving the accuracy and semantic coherence of the retrieval results. By directly outputting retrieval results based on the acquired target cache data, an end-to-end response process from pre-caching to obtaining retrieval results is achieved, simplifying the processing chain of the question-answering system. Therefore, this application effectively addresses the requirements of low latency, high accuracy, and semantically complete knowledge question answering.

[0075] See Figure 2The diagram illustrates a flowchart of a semantic unit generation method provided in an embodiment of this application, which may include the following steps:

[0076] S201. Perform sentence-level segmentation on the text content in the knowledge base to obtain a sentence sequence.

[0077] Sentence-level segmentation refers to dividing continuous text content stored in a knowledge base into independent sentence units according to the sentence boundaries of natural language. Sentence boundaries are typically identified by punctuation marks such as periods, question marks, exclamation marks, and semicolons. The resulting sentence sequence is a set of sentences arranged in their original order, with each sentence serving as an element in the sequence. This sequence preserves the original reading order of the text, providing a sequential basis for subsequent calculations of the semantic relationships between adjacent sentences.

[0078] The electronic device's processing system acquires raw text content from a knowledge base. This text content can be user-uploaded documents (supporting portable document formats such as PDF, Word documents, plain text format TXT, and markup language format Markdown), or text fragments already stored in the knowledge base. Preprocessing of the text content includes removing invalid characters and standardizing punctuation. Using natural language processing technology, sentence boundaries are identified, and the text content is segmented into independent sentences. For example, when a period, question mark, or exclamation mark indicating the end of a sentence is detected, the system determines the currently accumulated character sequence as a complete sentence and begins recording the next sentence. After the above processing, the sentence sequence corresponding to the raw text content is obtained, represented as S = {s1, s2, ..., s...}. n}, where n is the total number of sentences contained in the text content.

[0079] S202. Semantically encode adjacent sentences in the sentence sequence to obtain the semantic feature vector corresponding to each sentence.

[0080] Semantic encoding refers to converting natural language sentences into numerical vector representations that characterize their semantic information. Through semantic encoding, natural language text, which is inherently incapable of direct mathematical operations, is transformed into fixed-dimensional vectors, allowing semantic relationships between sentences to be quantified and calculated via vector operations. A semantic feature vector is a numerical vector obtained after semantic encoding of each sentence; this vector represents the sentence's semantic features in a high-dimensional space. Sentences with similar semantics have semantic feature vectors that are closer together in high-dimensional space; sentences with dissimilar semantics have semantic feature vectors that are farther apart.

[0081] In one embodiment of this application, a sentence-based bidirectional encoder representation (Sentence-BERT) model can be used to encode each sentence. This model is optimized for sentence-level semantic similarity calculation tasks based on the bidirectional encoder representation (BERT) model, and can generate high-quality sentence semantic representations.

[0082] For example, for each sentence s in the sentence sequence i When input into the Sentence-BERT model, the model outputs a corresponding 768-dimensional semantic vector, which is represented as follows:

[0083]

[0084] Among them, ℝ 768 Let b(s) represent a 768-dimensional real vector space. i ) is the sentence s i The semantic feature vector. Each dimension of this 768-dimensional vector carries feature information of the sentence in different semantic dimensions. By integrating information from all dimensions, this vector can represent the semantic content of the sentence relatively completely.

[0085] S203. Based on the semantic feature vectors of adjacent sentences, determine the semantic similarity of adjacent sentences, and define the semantic similarity as the semantic relevance of adjacent sentences.

[0086] Semantic similarity refers to the degree of semantic similarity between two adjacent sentences. Higher semantic similarity indicates that the two sentences are more related to each other and should belong to the same semantic unit; lower semantic similarity indicates that the two sentences have shifted or changed their content and may belong to different semantic units. In this embodiment, semantic relevance is quantified using semantic similarity.

[0087] After obtaining the semantic feature vectors of adjacent sentences, the similarity between them is calculated. For example, cosine similarity can be calculated as the semantic relevance between adjacent sentences. Cosine similarity is a measure of the consistency of the directions of two vectors. Its value range is [0, 1]. The closer the value is to 1, the more consistent the directions of the two vectors are, that is, the more similar the semantics of the two sentences are.

[0088] For example, for adjacent sentences s i and s i+1 The formula for calculating their semantic similarity is:

[0089] .

[0090] in, The dot product of semantic vectors, The similarity is the square root of the semantic vector. The value range is [0, 1]. The closer the value is to 1, the stronger the semantic relationship between the two sentences.

[0091] Through the above calculation, the system obtains a semantic similarity value for each pair of adjacent sentences, which is in the range of [0, 1]. This value is the semantic relevance of the adjacent sentences.

[0092] S204. Based on semantic relevance, determine the semantic unit division boundaries in the sentence sequence.

[0093] Semantic unit delimitation boundaries refer to the positions in a sentence sequence used to separate different semantic units. When the semantic correlation between adjacent sentences is low, it indicates a significant shift or change in the semantics between the two sentences, and this position is suitable as a semantic unit delimitation boundary.

[0094] In one embodiment of this application, the semantic unit partitioning boundary can be determined by the relationship between semantic relevance and a preset threshold. For example, a semantic similarity threshold γ can be preset, such as γ ranging from [0.6, 0.7]. When adjacent sentences s i and s i+1 Semantic similarity between Sim(s) i , s i+1 When the value is less than the threshold γ, the system determines the position between the two sentences as the semantic unit partition boundary. That is, sentence s i The last sentence belonging to the current semantic unit, sentence s i+1 The first sentence belonging to the next semantic unit. In addition to the semantic relevance threshold condition, the partition boundary can also be determined based on the length constraint of the semantic unit. The specific implementation of the length constraint will be further explained in subsequent embodiments.

[0095] S205. Divide the sentence sequence based on the semantic unit division boundary to obtain at least one semantic unit.

[0096] The processing system of an electronic device can traverse the obtained sentence sequence S = {s1,s2,……,s} n Group the sentences as follows:

[0097] First, initialize the current block word counter count = 0, and the current block sentence set Chunk. tmp = [ ]. Iterate through each sentence s in order. i Calculate the word length of the sentence. .

[0098] For the sentence s currently being traversed i The processing system of an electronic device determines the following two conditions:

[0099] Condition 1: s i After adding the current semantic unit, does the cumulative word length exceed the model's maximum context window length L? max That is, to judge Whether it is valid or not.

[0100] Condition 2, the current sentence s i Compared to the previous sentence s i-1 Whether the semantic similarity between them is less than the threshold γ, i.e., to determine Whether it is valid or not.

[0101] like That is, if condition one is met, add s. i If the length limit is not exceeded, then s i Join Chunk tmp Set up and update .

[0102] like or Then Store it as a new complete block in the block set and reset. ,Will Reset to an empty set. Where L max The threshold γ is dynamically adjusted based on the maximum number of context lexical units supported by the language model. The larger the context length supported by the language model, the fewer semantic units are required, and the more complete the semantic information of the source text can be preserved. The threshold γ ranges from [0.6, 0.7], and in this embodiment, γ = 0.7 is used.

[0103] Repeat the above judgment and operation until all sentences are segmented, resulting in a semantic unit set Chunk = {C1, C2, …, C t}, where t is the total number of semantic units obtained from the partition.

[0104] The semantic unit acquisition method provided in this embodiment performs sentence-level segmentation on the text content in the knowledge base, decomposing continuous text into independent sentence sequences. This allows subsequent processing to perform refined semantic analysis based on sentences as the basic unit, solving the semantic truncation problem caused by fixed-length units. By semantically encoding adjacent sentences in the sentence sequence, semantic feature vectors of each sentence are obtained, converting natural language text into a computable numerical vector representation. This enables the semantic relationships between sentences to be quantitatively evaluated through mathematical operations, providing a data foundation for the accurate calculation of semantic relevance. By determining the semantic similarity of adjacent sentences based on their semantic feature vectors and using this as the semantic relevance, the semantic closeness between sentences is quantified using metrics such as cosine similarity. This allows the system to objectively and accurately determine whether two sentences belong to the same semantic unit, rather than relying on fixed, manually set rules. By determining the semantic unit division boundary in the sentence sequence based on semantic relevance, and dividing the sentence sequence into semantic units based on the division boundary, the boundary position of the semantic unit corresponds to the semantic turning point of the text content. The sentences within each semantic unit are highly related semantically, while different semantic units are relatively independent semantically. This achieves dynamic semantic division, which is different from the fixed-length segmentation in the usual processing mode. It effectively maintains the semantic integrity of the original text content and avoids the semantic breakage and information fragmentation problems caused by segmentation.

[0105] Furthermore, in this embodiment of the application, the process of determining the semantic unit division boundary in the sentence sequence based on semantic relevance may include: in response to the semantic relevance between adjacent sentences being less than a preset threshold, or in response to the cumulative length of the current semantic unit reaching a preset length threshold, determining the current position as the semantic unit division boundary.

[0106] The preset threshold refers to a pre-defined semantic similarity threshold used to determine whether adjacent sentences should be considered as the boundary of a semantic unit. When the semantic relevance of adjacent sentences is lower than this threshold, it indicates that the semantics of the two sentences have changed significantly, and the corresponding position is suitable as a dividing boundary.

[0107] The preset length threshold refers to the maximum context window length that a language model can process, expressed in units of lexical units. When the accumulated number of lexical units in the current semantic unit reaches this length threshold, the semantic unit needs to be divided at this point, even if the semantics are not yet fully complete, to avoid exceeding the processing capacity limit of the language model.

[0108] Specifically, when performing semantic unit segmentation, the processing system of an electronic device can simultaneously monitor the two conditions mentioned above to determine whether to set a segmentation boundary at the current sentence position. Specifically, when determining based on semantic relevance, the semantic similarity between the currently traversed sentence and the previous sentence can be calculated in real time. When this similarity is less than a preset threshold, the current position is determined as the semantic unit segmentation boundary, indicating that the current sentence and the previous sentence have a weak semantic connection and should belong to different semantic units.

[0109] When making judgments based on cumulative length, a cumulative word counter for the current semantic unit can be maintained to record the total number of words already contained in the current semantic unit. Before adding the current sentence to the semantic unit, the cumulative word length after adding the sentence is pre-calculated. If the cumulative word length after addition exceeds a preset length threshold, the current position is determined as the semantic unit division boundary, and the current sentence is used as the first sentence of the next semantic unit.

[0110] When either of the above two conditions is met, the currently accumulated sentences can be saved as a complete semantic unit, and a new semantic unit can be started.

[0111] In this embodiment of the application, the process of dividing a sentence sequence based on semantic unit division boundaries to obtain at least one semantic unit includes: dividing the sentence sequence based on semantic unit division boundaries to obtain initial semantic units; and performing boundary overlap processing on adjacent initial semantic units so that the latter semantic unit includes the tail information of the former semantic unit.

[0112] Specifically, the processing system of the electronic device traverses each sentence in the sentence sequence, determining when to end the current semantic unit and start a new semantic unit based on boundary judgment conditions. After traversal, an initial set of semantic units is obtained, each initial semantic unit containing several semantically related sentences, and the word length of each semantic unit does not exceed a preset length threshold.

[0113] After the initial partitioning is completed, the electronic device's processing system performs boundary overlap processing on adjacent initial semantic units. For two adjacent semantic units (Chunks)... m and Chunk m+1 From Chunk m Extract a certain percentage of lexical content from the end of the string and add this lexical content to the Chunk. m+1 The beginning part.

[0114] For example, to avoid semantic breaks caused by segmenting sentences with strong semantic connections due to length limitations, a sliding window is introduced between sentence segments:

[0115]

[0116] in , i.e., from Chunk m Extract the last 10% of the word sequence by length, Token end-k (Chunk m ) indicates from Chunk m Extract the last k tokens from the end, Chun km+1, origin For Chunk m+1 The original content. By using a sliding window, each block will have the last 10% of the tokens from the previous block, forming a semantic bridge and preventing semantic breaks.

[0117] This embodiment divides the sentence sequence into initial semantic units based on semantic unit division boundaries. Then, it performs boundary overlap processing on adjacent initial semantic units so that the later semantic unit contains the tail information of the previous semantic unit. This makes the semantic information that was originally separated by the division boundaries form a smooth transition at the boundary, avoiding the semantic break across units caused by strict division. This ensures that users can still obtain coherent semantic information when querying boundary areas, further improving the completeness and accuracy of the search results.

[0118] This application also provides a method for retrieving key-value cache, which may include the following steps:

[0119] S301. Vectorize each semantic unit to obtain the embedded representation of each semantic unit.

[0120] Vectorization refers to converting natural language text within semantic units into a sequence of numerical vectors that a language model can process. A semantic unit consists of multiple sentences, and each sentence is composed of multiple tokens. Vectorization requires generating a corresponding embedding vector for each token.

[0121] Embedding representation refers to the vector sequence obtained by semantic units after word embedding and positional encoding. Word embedding is used to represent the semantic information of each word, and positional encoding is used to represent the sequential position of each word in the semantic unit. Embedding representation is the input data form of the self-attention layer of a language model.

[0122] The processing system of electronic devices uses semantic units C m Lexical segmentation breaks down continuous text into the smallest units that a language model can recognize, known as lexical units. A lexical unit can be a complete word or a sub-word, depending on the segmentation method used by the language model.

[0123] Lexical embedding is performed on each lexical unit. Lexical embedding is the operation of converting the discrete identifiers of lexical units into continuous vectors. Internally, the language model can maintain a lexical embedding matrix, where each row corresponds to the embedding vector of a lexical unit in the vocabulary. Based on the index of each lexical unit in the vocabulary, the corresponding embedding vector is found in the lexical embedding matrix to obtain the lexical embedding representation of each lexical unit.

[0124] Positional encoding is performed on each word. Since the self-attention calculation of the language model itself does not contain sequence order information, additional positional encoding is needed to represent the position of the word in the sequence. Positional encoding can be a fixed-form encoding function or a learnable positional embedding matrix. A corresponding positional encoding vector is generated based on the position index of each word in the semantic unit. The word embedding vector of each word is added element-wise to the positional encoding vector to obtain the final embedding vector of each word. The embedding vectors of all words are arranged in order to obtain the semantic unit C. m Embedding representation matrix Emb(C) m The dimension of this matrix is ​​k. m ×d model , where k m For semantic unit C m The total number of lexical units contained, d model is the dimension of the hidden layer of the language model.

[0125] S302. Based on the key projection matrix and value projection matrix of the language model, perform projection transformation on the embedding representation to generate the key cache matrix and value cache matrix for each semantic unit.

[0126] Here, the key projection matrix and value projection matrix refer to the linear transformation matrices used in the language model's attention layer to project the input vector into the key space and value space, respectively. The key projection matrix converts the input vector into a key vector, used for similarity calculation with the query vector; the value projection matrix converts the input vector into a value vector, used for weighted summation based on the attention weights.

[0127] The key cache matrix refers to the sequence of key vectors obtained after transforming semantic units through the key projection matrix, which is used for subsequent similarity matching with query requests. The value cache matrix refers to the sequence of value vectors obtained after transforming semantic units through the value projection matrix, which is used for subsequent generation of context representation based on the matching results.

[0128] Specifically, the processing system of an electronic device can be based on the key projection matrix W of the self-attention layer of a language model. K Sum projection matrix W V The resulting embedding representation Emb(C) m Perform projection transformation.

[0129] The key projection transformation is calculated as follows: the embedding representation matrix Emb(C) is transformed into a key projection matrix. m ) and the key projection matrix W K Perform matrix multiplication to obtain the key cache matrix K. m For example, the calculation formula is:

[0130] .

[0131] The dimension of the key cache matrix is ​​k. m ×d K , where k m d represents the number of lexical units in a semantic unit. K is the dimension of the key vector.

[0132] The value projection transformation is calculated as follows: the embedding representation matrix Emb(C) is transformed into a single matrix. m ) and the projection matrix W V Perform matrix multiplication to obtain the value cache matrix V. m Its calculation formula can be expressed as:

[0133] .

[0134] The dimension of this value cache matrix is ​​k. m ×d V , where k m d represents the number of lexical units in a semantic unit. V is the dimension of the value vector.

[0135] Key projection matrix W K Sum projection matrix W V These are the parameter matrices learned by the language model during training. They map the input vector from the hidden layer dimensional space to the key space and value space, respectively. Through these two projection transformations, the semantic information of the semantic unit is encoded into a key vector form suitable for similarity calculation and a value vector form suitable for information aggregation, respectively.

[0136] S303. Concatenate the key cache matrix and value cache matrix of each semantic unit to obtain the global key-value cache corresponding to the text content in the knowledge base.

[0137] Global key-value caching refers to the complete cached data obtained by concatenating the key cache matrix and value cache matrix corresponding to all text content in the knowledge base according to the semantic unit order in the original text. The global key-value cache contains the key and value information of all text content in the knowledge base, serving as a complete data source for subsequent query matching.

[0138] The processing system of an electronic device performs row concatenation on the key cache matrix of all semantic units according to the order in which they appear in the original text content, to obtain the global key cache matrix K.global The calculation formula for this splicing operation can be:

[0139] .

[0140] in, The total number of terms in all text content in the knowledge base, where Concat is the line concatenation function, and d k is the dimension of the key vector.

[0141] The value cache matrices of each semantic unit are concatenated row by row to obtain the global value cache matrix V. global The corresponding calculation formula for this splicing operation is:

[0142] .

[0143] Where N is the total number of lexical units in all text content in the knowledge base, and d V is the dimension of the value vector.

[0144] The specific method of the concatenation operation is as follows: For the key cache matrix K1, K2, ..., K t They are arranged in row order, with all rows of the key cache matrix of the first semantic unit first, all rows of the key cache matrix of the second semantic unit after the first semantic unit, and so on, eventually forming a large matrix K. global Global value cache matrix V global The concatenation method is similar. Global key cache matrix K global and global value cache matrix V global Together, they form a complete cached representation of the knowledge base text content, which does not need to be recalculated during subsequent queries.

[0145] S304. Establish an index association between text content, semantic units, and storage locations in the global key-value cache, and store the target cache library for the values.

[0146] After concatenating the global key-value cache, record the start and end row positions of the key cache matrix and value cache matrix corresponding to each semantic unit in the global cache.

[0147] For each semantic unit C m Record the following information: the identifier of the semantic unit, and the starting row index of the semantic unit in the global key cache. m The end-line index of this semantic unit in the global key cache. m Since the key cache matrix and the value cache matrix are aligned in the row dimension, meaning that the j-th row in the key cache matrix corresponds to the same tag as the j-th row in the value cache matrix, the same semantic unit has the same start and end row indices in both the key cache and the value cache.

[0148] The above information is organized into an index data structure. This index structure takes the form of a mapping from document identifiers to semantic unit identifiers, and a mapping from semantic unit identifiers to the start and end indexes of the cache storage location. Specifically, it is represented as: Index = {DocID: ChunkIDs: [(Chunk1, start1, end1), (Chunk2, start2, end2), ...,(Chunk...}} t , start t end t )]}. Among them, start m end m Block C m The starting and ending row indices of the corresponding KV cache in the global cache.

[0149] After completing the construction of the index association, the global key cache matrix K is... global Global value cache matrix V global The index-related data is then stored in the target cache. The target cache can be implemented using a distributed caching database such as Redis or Memcached to support high-concurrency read access. Once storage is complete, the text content in the knowledge base has been transformed from raw text into a key-value cache, ready to respond to user query requests at any time.

[0150] The key-value caching method provided in this application obtains the embedded representation of each semantic unit by vectorizing it, converting natural language text into a numerical vector sequence that can be processed by a language model, thus providing a standardized input data format for subsequent projection transformations. It generates key and value cache matrices for each semantic unit by projecting the embedded representations using key and value projection matrices based on the language model, encoding the semantic information of each semantic unit into key vectors suitable for similarity calculations and value vectors suitable for information aggregation, thus preparing directly usable cached data for query matching and result generation. Finally, it obtains the global key-value cache matrix corresponding to the text content by concatenating the key and value cache matrices of each semantic unit. Key-value caching integrates cached data of all text content in the knowledge base into a unified storage structure, facilitating unified access during subsequent queries. By establishing an index association between text content, semantic units, and storage locations in the global key-value cache and storing them in the target cache library, it achieves rapid location and persistent storage of cached data. This allows text content in the knowledge base to be preprocessed only once and reused for all subsequent queries, eliminating the need to repeatedly perform vectorization and projection transformation operations during each query, thus reducing computational overhead during queries. Unlike ordinary key-value caching reuse technologies where the cache is calculated in real time from the current conversation, the key-value cache in this application is pre-calculated and stored in the target cache library before the query request arrives, supporting reuse across documents and sessions.

[0151] In this embodiment of the application, the process of retrieving target cache data matching the query request from the target cache library may include the following steps:

[0152] S401. The query request is vectorized to obtain the embedded representation of the query request.

[0153] A query request refers to a natural language question or instruction input by the user. Vectorization processing refers to converting the query request text into a sequence of numerical vectors that the language model can process, similar to the processing of semantic units in the above embodiments, and will not be detailed here. The embedded representation of the query request refers to the vector sequence obtained after the query request has undergone lexical embedding and positional encoding, which is the input data for subsequent projection transformation.

[0154] The query request is lexicalized, dividing the query request text into the smallest units that the language model can recognize, namely, a sequence of terms. For example, a query request containing p terms can be represented as Q = {q1, q2, …, q}. pFor each lexical unit, lexical embedding is performed. Based on the index of each lexical unit in the language model's vocabulary, the corresponding embedding vector is found from the lexical embedding matrix to obtain the lexical embedding representation of each lexical unit. Position encoding is performed on each lexical unit. Based on the position index of each lexical unit in the query request, a corresponding position encoding vector is generated.

[0155] The word embedding vector of each word is added element-wise to the positional encoding vector to obtain the final embedding vector of each word. The embedding vectors of all words are then arranged in order to obtain the embedding representation matrix Emb(Q) of the query request. The formula for calculating this matrix is:

[0156] .

[0157] Where TokenEmb is the built-in lexical embedding function of the language model, PosEmb is the positional encoding function, p is the number of lexical units in the query request, and d model This represents the hidden layer dimension of the language model.

[0158] S402. Based on the language model, the query projection matrix is ​​obtained by projecting the embedded representation of the query request.

[0159] The query projection matrix is ​​a linear transformation matrix used in the language model's self-attention layer to project the input vector into the query space. After transformation by the query projection matrix, the embedded representation of the query request is converted into the query projection matrix, where each query element is used to calculate similarity with the key vector in the key cache matrix.

[0160] The query projection transformation is calculated by combining the embedding representation matrix Emb(Q) with the query projection matrix W. Q Perform matrix multiplication to obtain the query projection matrix, the formula for which is:

[0161] .

[0162] Among them, W Q Here, d represents the query projection matrix of the language model's self-attention layer, p is the number of lexical units in the query request, and d is the number of lexical units in the query request. Q The dimension of the query vector.

[0163] Query projection matrix W Q This is the parameter matrix learned by the language model during training, which maps the input vector from the hidden layer dimensional space to the query space. After projection transformation, each word in the query request is converted into a query vector, which is used for subsequent similarity calculation with the key vectors in the key cache matrix.

[0164] S403. Calculate the similarity score between each query element in the query projection matrix and each cache element in the key-value cache of the target cache library.

[0165] The query element refers to the projection matrix Q. proj Each row in the matrix represents the query vector corresponding to each query term. A cache element refers to the global key cache matrix K in the target cache library. global Each row in the table represents the key vector corresponding to each cached term. The similarity score refers to the semantic relevance between the query element and the cached element; a higher score indicates a greater semantic relevance.

[0166] S404. Based on similarity scores, determine the target cached data that matches the query request from the key-value cache.

[0167] The processing system of an electronic device can filter cached elements that match a query request from a global key-value cache based on similarity scores. The filtering method can be based on preset matching rules, such as selecting the top few cached elements with the highest similarity scores, or selecting all cached elements with similarity scores exceeding a certain threshold. After determining the target cached data, the corresponding key-cache data and value-cache data can be located and extracted from the target cache library based on the established index associations, serving as input for generating subsequent search results.

[0168] This embodiment vectorizes the query request to obtain its embedded representation, converting the user's natural language query into a numerical vector sequence that the language model can process, thus preparing standardized input data for subsequent projection transformation and similarity calculation. A query projection matrix is ​​obtained by projecting the embedded representation of the query request using a query projection matrix based on the language model. This encodes the semantic information of the query request into a query vector form suitable for similarity calculation with the key cache, enabling the matching of the query request and the knowledge base cached data to be performed in a unified vector space. Finally, the matching is performed by calculating the relationship between each query element in the query projection matrix and each cache element in the key-value cache of the target cache. The similarity score quantifies the semantic relevance between the query request and each cached term in the knowledge base, providing an objective numerical basis for filtering target cached data. By determining the target cached data matching the query request from the key-value cache based on the similarity score, it achieves accurate location of cached content semantically related to the query request from the pre-built global cache. This eliminates the need to re-access the original knowledge base or perform real-time vector retrieval operations during the query. Unlike typical retrieval enhancement generation methods that require real-time approximate nearest neighbor search in the vector database, this application can directly complete the matching from the pre-built cache through similarity calculation, effectively reducing query response latency.

[0169] Furthermore, the process of calculating the similarity score between each query element in the query projection matrix and each cached element in the key-value cache of the target cache library in this embodiment may include the following steps:

[0170] S501. Based on the target dimension set, calculate the similarity sub-parameters between each query element in the query projection matrix and each cache element in the key-value cache of the target cache library.

[0171] The target dimension set refers to multiple evaluation dimensions used to assess the semantic relevance between query elements and cached elements. Each dimension characterizes semantic similarity from a different perspective, and the combined evaluation results of multiple dimensions can more comprehensively reflect the semantic relationship between the two elements.

[0172] Specifically, this target dimension set can include a first dimension, a second dimension, and a third dimension. The first dimension characterizes semantic direction consistency, i.e., the degree of similarity between two vectors in a direction. This dimension assesses whether the semantic reference of the query element and the cached element is consistent. For example, if "apple" in the query refers to the meaning of fruit, and "apple" in the cache also refers to the meaning of fruit, then the direction consistency is high. The second dimension characterizes semantic matching strength, i.e., the strength of the match between two vectors in a specific direction. This dimension can amplify the matching signal between strongly related terms, giving higher weight to highly related term pairs. The third dimension characterizes semantic detail differences, i.e., subtle differences in local features between two vectors. This dimension is more sensitive to technical terms and fixed collocations, and can extract precise matching requirements at the term level.

[0173] The similarity sub-parameter refers to the similarity metric value calculated independently on each dimension. Each sub-parameter reflects the degree of similarity between the query element and the cached element from a specific perspective.

[0174] In one embodiment of this application, for querying the projection matrix Q proj Each query element q in i and the global key cache matrix K global Each cache element k in j The system calculates similarity sub-parameters from the first, second, and third dimensions, respectively.

[0175] When calculating the similarity sub-parameter in the first dimension, the system calculates the cosine similarity between the query element and the cached element. Cosine similarity measures the consistency of direction between two vectors, and its formula is the dot product of the two vectors divided by the product of their magnitudes; its value ranges from 0 to 1. The closer the cosine similarity is to 1, the more consistent the directions of the two vectors are, and the higher the semantic directional consistency. The formula for calculating the similarity sub-parameter in the first dimension is:

[0176] .

[0177] Where, q i To query the projection vector of the i-th query element in the projection matrix, i.e., a single query term, k j The key vector of the j-th cache element in the global cache key cache matrix, i.e., a single cached term, is used to quantify the semantic similarity between vectors.

[0178] When calculating the similarity sub-parameter from the second dimension, the system calculates the dot product similarity between the query element and the cached element. The dot product similarity is calculated by dividing the dot product of the two vectors by the square root of the query vector's dimension. This calculation method is used in the attention mechanism to scale the dot product result to prevent gradient vanishing. Dot product similarity can strengthen the response intensity of strongly correlated signals between vectors. The formula for calculating the similarity sub-parameter from the second dimension is:

[0179] .

[0180] Where, d k denoted as the dimension of the key vector, this formula is used to strengthen the scoring of strongly related terms.

[0181] When calculating the similarity sub-parameter from the third dimension, the system calculates the distance similarity between the query element and the cached element. The distance similarity is calculated as follows: the absolute difference between the two vectors in each dimension is calculated, the sum of the absolute differences across all dimensions is divided by the vector dimension to obtain the mean absolute difference, and then 1 is subtracted from this mean absolute difference to obtain the distance similarity. This value reflects the closeness of the two vectors in local features and is more sensitive to matching technical terms and fixed collocations. The formula for calculating the similarity sub-parameter of the third dimension is:

[0182] .

[0183] Where, d k Let q be the dimension of the key vector. i [t] and k j [t] represents the values ​​of the query element and the cached element on the t-th dimension, respectively. This formula is used to capture the differences in local semantic details of word elements and is more sensitive to technical terms and fixed collocations.

[0184] After calculating the three similarity sub-parameters, three values ​​are obtained, each ranging from 0 to 1.

[0185] S502. The similarity sub-parameters are fused to obtain the similarity score.

[0186] The similarity sub-parameters are fused to form a comprehensive score by combining the similarity sub-parameters from multiple dimensions according to certain weights. The purpose of fusion is to integrate the evaluation results of each dimension to obtain a single score that can comprehensively reflect the semantic relevance between the query element and the cached element.

[0187] For example, the processing system of an electronic device assigns a weight coefficient α to the similarity sub-parameter of the first dimension, a weight coefficient β to the similarity sub-parameter of the second dimension, and a weight coefficient γ to the similarity sub-parameter of the third dimension, and the sum of the three weight coefficients is 1.

[0188] The overall similarity score is obtained by multiplying the similarity sub-parameters of each dimension by their corresponding weight coefficients and then summing the results. The formula for calculating the overall similarity score is as follows:

[0189] .

[0190] Wherein, α, β, and γ are the scoring weights, and α + β + γ = 1. In this embodiment, the weight coefficients α, β, and γ can be 0.4, 0.3, and 0.3 respectively, meaning that cosine similarity is used as the main influencing factor for scoring. The three weight coefficients can be adjusted according to different application scenarios and language model characteristics.

[0191] The final similarity score ranges from 0 to 1. The closer the value is to 1, the stronger the semantic relevance between the query element and the cached element.

[0192] This embodiment calculates similarity sub-parameters between each query element in the query projection matrix and each cached element in the key-value cache of the target cache library based on a target dimension set. The target dimension set includes at least a first dimension representing semantic direction consistency, a second dimension representing semantic matching strength, and a third dimension representing semantic detail differences. It evaluates the similarity between query elements and cached elements from three different perspectives: semantic direction, matching strength, and detail differences. Each dimension focuses on different directions of semantic similarity, making the evaluation results more comprehensive and detailed. A similarity score is obtained by fusing the similarity sub-parameters. The similarity sub-parameters of each dimension are weighted and summed according to preset weight coefficients. The combined evaluation results of each dimension form a single score that comprehensively reflects the semantic relevance between query elements and cached elements. Compared to methods relying solely on a single similarity calculation method, this embodiment reduces the potential bias of single-dimensional evaluation through multi-dimensional fusion. For example, cosine similarity is insensitive to vector magnitude, while dot product similarity is significantly affected by vector magnitude. Weighted fusion can compensate for each other's shortcomings, thereby improving matching accuracy and providing a more reliable basis for subsequently determining target cache data.

[0193] In this application, the process of determining the target cached data matching the query request from the key-value cache based on similarity scores may include the following steps:

[0194] S601. Determine the dynamic matching threshold based on the distribution characteristics of similarity scores.

[0195] The distribution characteristics of similarity scores refer to the statistical properties of the similarity scores between all query elements and all cached elements, such as the mean and standard deviation of the scores. The mean reflects the baseline level of overall similarity, and the standard deviation reflects the dispersion of the score distribution. The dynamic matching threshold is a threshold adaptively calculated based on the score distribution characteristics of the current query request. This threshold is not a fixed constant but dynamically adjusted according to different query requests.

[0196] The processing system of the electronic device first calculates the average μ and standard deviation θ of all the obtained similarity scores.

[0197] The average value is calculated by summing the similarity scores between all query elements and all cached elements, then dividing by the total number of comparisons. The total number of comparisons equals the number of query terms p multiplied by the total number of terms N in the knowledge base. The formula for calculating the average value μ is:

[0198] .

[0199] The standard deviation is calculated by taking the square of the deviation of each similarity score from the mean, summing all the squared deviations, dividing by the total number of comparisons, and then taking the square root. The standard deviation reflects the degree of fluctuation in similarity scores around the mean. The formula for calculating the standard deviation θ is:

[0200] .

[0201] The dynamic matching threshold τ is calculated based on the mean and standard deviation. The formula for calculating the dynamic matching threshold is:

[0202] .

[0203] Where ε is the adjustment coefficient. In this embodiment, the adjustment coefficient ε is 0.65. The dynamic matching threshold τ calculated by this formula ranges from 0 to 1. When the overall score distribution is high, the threshold is increased accordingly; when the overall score distribution is low, the threshold is decreased accordingly. This adaptive adjustment method allows the matching conditions to adapt to the characteristics of different query requests.

[0204] S602. Cache elements whose similarity scores meet the dynamic matching threshold are identified as target cache data.

[0205] Meeting the dynamic matching threshold means that the similarity score of a cached element is greater than or equal to the calculated dynamic matching threshold τ. Cached elements that meet this condition are considered to have sufficient semantic relevance to the query request and are selected as target cached data.

[0206] For each cached element, check its similarity score with each query element. For each cached element k j The system records its relationship with all query elements q. i Similarity score Score(q) i , k j If at least one query element results in a similarity score greater than or equal to the dynamic matching threshold τ, then the cached element is considered a match, and the match flag Hit(j) = 1; otherwise, Hit(j) = 0. All matched cached elements constitute the target cached data set.

[0207] Furthermore, based on the target cached data, outputting the target retrieval results corresponding to the query request may include the following steps:

[0208] S701. Based on the similarity score, determine the attention weight corresponding to the target cached data.

[0209] Attention weight refers to the proportion of contribution each hit cache element makes in generating the context representation of the query request. Cache elements with higher similarity scores receive higher attention weights.

[0210] For each query element q i Attention weights are calculated only for cached elements that are hit. The attention weights are calculated as follows: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] i With each of the cached elements k that were hit j The similarity scores are proportionally normalized so that the sum of all weights is 1.

[0211] The normalization process involves using the similarity score of each hit cache element as input to an exponential function, then dividing the result by the sum of the similarity scores of all hit cache elements to obtain the attention weight for each cache element. The formula for calculating the attention weight is:

[0212] .

[0213] Here, I is an indicator function that returns 1 if the Hit condition is met, and 0 otherwise. This is a scaling factor to prevent the function gradient from vanishing due to excessively high similarity scores.

[0214] S702. Based on attention weights, perform weighted aggregation on the value cache matrix corresponding to the target cache data to generate a context representation of the query request.

[0215] Weighted aggregation refers to the weighted summation of the value vectors corresponding to the hit cached elements based on attention weights. The result of weighted aggregation is a contextual representation of each term in the query request, which incorporates query-related information from the knowledge base. Contextual representation, which is a vector representation that integrates semantic information from the query request and relevant information from the knowledge base, forms the basis for the input language model to generate the final answer.

[0216] Specifically, for each query element q i The system performs a weighted summation of the corresponding attention weight with the value vectors of each cached element in the target cache data. During the weighted summation, each value vector is multiplied by its corresponding attention weight, and then all products are summed to obtain the context representation of the query element (Output). i .

[0217] Query element q i The context of Output i The calculation formula is:

[0218] .

[0219] Where Attn(q) i , K global V is the attention weight vector. global For global value cache matrix, d represents the transpose of the global value cache matrix. V is the dimension of the value vector.

[0220] The context representations of all query elements are concatenated in order to obtain the complete inference input vector Output, where, Concat indicates concatenating parameters.

[0221] S703. Input the context representation into the language model to generate the target retrieval results.

[0222] A language model refers to a deep learning model used to generate natural language text, such as a large-scale language model based on a transformer architecture used in this embodiment. The language model includes components such as feedforward networks and decoding layers, used to convert contextual representations into natural language responses.

[0223] Specifically, the inference input vector Output obtained by the processing system of the electronic device is input into the feedforward network of the language model for processing, and then the response text is generated word by word through the decoding layer.

[0224] The feedforward network performs a non-linear transformation on the context representation to extract higher-level feature representations. The decoding layer, based on the output of the feedforward network, generates the answer text word by word in an autoregressive manner. That is, it generates one word at a time, then uses that word as input to continue generating the next word, until a complete answer is generated or a stopping condition is met. The final generated answer, Answer, is the target retrieval result. Its generation process can be represented as: Answer = Decode(FFN(Output)), where FFN is the feedforward network and Decode is the decoding layer.

[0225] The target cache data determination and retrieval result generation method provided in this application determines a dynamic matching threshold based on the distribution characteristics of similarity scores, enabling the matching threshold to adaptively adjust according to the score distribution of the current query request. Compared with the fixed threshold method, this method has better adaptability and robustness. By determining cache elements whose similarity scores meet the dynamic matching threshold as target cache data, it achieves accurate screening of cache elements semantically related to the query request, eliminating interference from irrelevant cache elements with low similarity. By determining the attention weight corresponding to each target cache data based on the similarity score, cache elements with higher semantic relevance receive greater contribution weight. By weighted aggregation of the value cache matrix corresponding to the target cache data based on the attention weight to generate the context representation of the query request, it integrates information related to the query request in the knowledge base into the semantic representation of the query request. By inputting the context representation into the language model to generate the target retrieval result, it realizes the conversion from context representation to natural language answer. The entire process only calculates the hit cache elements, avoiding the computational overhead of attention calculation on the full cache, improving inference efficiency and reducing the first character response time.

[0226] In some embodiments of this application, the information retrieval method further includes:

[0227] S801. Based on the association frequency of historical queries and semantic units, construct a target query association graph.

[0228] A target query association graph is a data structure that records the relationships between historical queries and semantic units. Each node represents a query or a semantic unit, the edges between nodes represent the historical records of queries hitting that semantic unit, and the weight of the edges represents the frequency of the hits.

[0229] The processing system of the electronic device records historical data for each query during operation. When a user initiates a query request and it is processed by this solution, the system records the semantic information of the query request and the identifier of the semantic unit hit by the query. The system maintains a query set QSet and a semantic unit set ChunkSet.

[0230] For each query Q S The system records which semantic units were hit by the query and counts the frequency of each semantic unit being hit by the query. The system constructs the association between the query and the semantic units, represented as a triple (Q). S Chunk t , ω), where ω is the query Q S Hit semantic unit Chunk t The frequency of.

[0231] Organize all historical queries and semantic units, along with the frequency of their associations, to form a target query association graph G, where, .

[0232] in, For historical query collection; For a block set; For query Hit the block The frequency of.

[0233] S802. In response to the query request, determine the historical queries that are semantically related to the query request from the target query association graph.

[0234] When the system receives the current query request Q, the system will associate Q with each historical query Q in the target query association graph. S Perform semantic similarity calculation. The method for calculating semantic similarity is the same as that for calculating semantic similarity between sentences, that is, converting the two queries into semantic feature vectors and then calculating the cosine similarity.

[0235] The formula for calculating similarity is:

[0236] .

[0237] Among them, Emb(Q) and Emb(Q) S ) are the semantic feature vectors of the current query and the historical query, respectively.

[0238] The system compares the calculated semantic similarity with a preset threshold. In this embodiment, the preset threshold is set to 0.75. If Sim(Q, Q) S If Q > 0.75, then the historical query Q is determined. S This historical query, semantically related to the current query Q, will be used for prefetching.

[0239] S803. Based on the target semantic unit corresponding to the historical query, cache the key value of the target semantic unit in the target storage area.

[0240] The target storage area refers to storage space with faster access speed, such as the cache or high-speed cache memory in memory.

[0241] For a given historical query Q that is semantically associated with the current query, S The system searches for semantic units associated with the historical query from the target query association graph. The system then ranks these units according to their association frequency ω(Q). S Chunk t The semantic units are sorted from high to low, and the top few semantic units with the highest association frequency are selected as prefetching objects. For example, in this embodiment, the number of prefetching objects is set to three.

[0242] Based on the established index associations, the system locates the storage location of the key-value cache for these prefetch semantic units in the target cache library and preloads this cached data into the target storage area. The target storage area can be a prefetch buffer in memory, which has a higher access speed than the regular access path of the target cache library.

[0243] S804. Retrieve the target cache data that matches the query request from the target storage area.

[0244] Upon receiving a query request, the electronic device's processing system prioritizes reading prefetched key-value cache data from the target storage area for matching calculations. Because the target storage area is accessed more quickly, reading the prefetched cache data has lower latency. For cached data not within the prefetch range, the system still reads it from the regular storage location of the target cache.

[0245] This embodiment constructs a target query association graph based on the association frequency of historical queries and semantic units, recording the matching relationships between queries and semantic units in the historical question-and-answer process in graph form, providing a data foundation for subsequent prefetching operations. In response to a query request, it identifies historical queries semantically related to the query request from the target query association graph, and uses the semantic similarity between historical queries and the current query to identify historical access patterns that may be related to the current query, enabling prefetching operations to predict based on actual usage patterns. It caches the key-value pairs of target semantic units corresponding to historical queries in the target storage area, and preloads the key-value pairs predicted to be matched by the current query into a faster storage area. By retrieving target cache data matching the query request from the target storage area, the data reading operation during the matching process can preferentially obtain data from the high-speed storage area, thereby reducing input / output reading latency and further reducing the first-word response time of the model's answer.

[0246] See Figure 3The diagram shown is an interactive illustration of an information retrieval system in an application scenario provided by an embodiment of this application. Taking an internal enterprise knowledge base question-and-answer scenario as an example, the information retrieval method of this application embodiment is described in general.

[0247] like Figure 3 As shown, the user first uploads knowledge base documents to the information retrieval system, such as a collection of documents containing employee handbooks, reimbursement policies, and technical specifications. After receiving the documents, the semantic coherence segmentation module processes the document content: this module performs sentence-level segmentation, semantically encodes adjacent sentences and calculates semantic similarity, dynamically divides boundaries based on semantic similarity thresholds and length thresholds, and performs boundary overlap processing on adjacent semantic units to obtain multiple semantically complete semantic units.

[0248] The KV cache pre-computation module vectorizes each semantic unit, generating key and value cache matrices for each unit based on the key and value projection matrices of the language model. These matrices are then concatenated to form a global key cache and a global value cache. An index association is established between documents, semantic units, and cache storage locations, thus completing the knowledge cache module. This module stores the semantic unit-level key-value caches corresponding to all documents in the knowledge base.

[0249] When a user initiates a knowledge base query, such as the query "How far in advance should I apply for annual leave?", the adaptive hit cache inference module receives the query request, performs vectorization and projection transformation on the query request to obtain a query projection matrix, calculates the similarity score between each query element in the query projection matrix and each cached element in the key-value cache of the knowledge caching module, determines a dynamic matching threshold based on the distribution characteristics of the similarity scores, identifies cached elements that meet the threshold as target cached data, and performs weighted aggregation on the value cache matrix corresponding to the target cached data based on attention weights to generate a context representation. Finally, the context representation is input into the language model to generate a knowledge answer, such as the system output "According to company regulations, annual leave applications must be submitted three working days in advance".

[0250] In this scenario, after a user uploads a knowledge base document, the system divides the document into semantically complete units using a semantic coherence segmentation module. A key-value cache is pre-built by the knowledge caching module, eliminating the need for real-time vector retrieval when a user initiates a query. The adaptive cache inference module directly matches relevant content from the knowledge cache and generates an answer. Compared to the common approach of requiring real-time retrieval for each query, this embodiment reduces the time users spend waiting for the first character response. Furthermore, because the cached content is built upon semantic integrity, the answers are more coherent and accurate, effectively improving the user experience of knowledge base question answering.

[0251] See Figure 4 This application also provides an information retrieval device, which includes:

[0252] The receiving unit 11 is used to receive a query request, which represents a request to retrieve information from text content in the knowledge base;

[0253] The matching unit 12 is used to obtain target cache data that matches the query request from the target cache library, wherein the target cache library is pre-built based on the text content in the knowledge base, and is used to store key-value caches that store at least one semantic unit corresponding to the text content in the knowledge base, wherein the semantic unit is obtained by dividing the text content based on the semantic correlation between sentences in the text content;

[0254] Output unit 13 is used to output the target retrieval result corresponding to the query request based on the target cache data.

[0255] It should be noted that the specific implementation of each unit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0256] In another embodiment of this application, a readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the information retrieval method as described above.

[0257] In another embodiment of this application, an electronic device is also provided, including at least one processor and a target prediction model capable of running on the processor. The processor is configured to perform the following operations: receiving a query request, the query request representing a request to retrieve information from text content in a knowledge base; obtaining target cache data matching the query request from a target cache library, wherein the target cache library is a key-value cache pre-built based on the text content in the knowledge base, used to store at least one semantic unit corresponding to the text content in the knowledge base, the semantic unit being obtained by dividing the text content based on the semantic correlation between sentences in the text content; and outputting the target retrieval result corresponding to the query request based on the target cache data.

[0258] It should be noted that the specific implementation of the processor in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0259] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0260] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0261] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0262] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An information retrieval method, comprising: Receive a query request, which represents a request to retrieve information from text content in a knowledge base; Obtain target cache data matching the query request from the target cache library, wherein the target cache library is a key-value cache pre-built based on the text content in the knowledge base, used to store at least one semantic unit corresponding to the text content in the knowledge base, and the semantic unit is obtained by dividing the text content based on the semantic correlation between sentences in the text content; Based on the target cached data, output the target retrieval result corresponding to the query request.

2. The method according to claim 1, wherein the semantic unit is obtained in the following manner: Sentence-level segmentation is performed on the text content in the knowledge base to obtain a sentence sequence; Semantic encoding is performed on adjacent sentences in the sentence sequence to obtain semantic feature vectors corresponding to each sentence; Based on the semantic feature vectors of adjacent sentences, the semantic similarity of the adjacent sentences is determined, and the semantic similarity is determined as the semantic relevance of the adjacent sentences; Based on the semantic relevance, the semantic unit division boundaries in the sentence sequence are determined; The sentence sequence is divided based on the semantic unit partitioning boundary to obtain at least one semantic unit.

3. The method according to claim 2, wherein determining the semantic unit partitioning boundary in the sentence sequence based on the semantic relevance includes: In response to the semantic correlation between adjacent sentences being less than a preset threshold, or in response to the cumulative length of the current semantic unit reaching a preset length threshold, the current position is determined as the semantic unit division boundary; The step of dividing the sentence sequence based on the semantic unit partitioning boundary to obtain the at least one semantic unit includes: The sentence sequence is divided based on the semantic unit partitioning boundaries to obtain initial semantic units; The adjacent initial semantic units are overlapped so that the latter semantic unit contains the tail information of the former semantic unit.

4. The method according to claim 1, wherein the key-value cache is obtained in the following manner: Each semantic unit is vectorized to obtain the embedded representation of each semantic unit; Based on the key projection matrix and value projection matrix of the language model, the embedding representation is projected and transformed to generate the key cache matrix and value cache matrix for each semantic unit. The key cache matrix and value cache matrix of each semantic unit are concatenated to obtain the global key-value cache corresponding to the text content in the knowledge base; Establish an index association between the text content, the semantic unit, and the storage location in the global key-value cache, and store it in the target cache library.

5. The method according to claim 1, wherein obtaining the target cache data matching the query request from the target cache library includes: The query request is vectorized to obtain an embedded representation of the query request; The query projection matrix is ​​obtained by projecting the embedded representation of the query request based on the language model. Calculate the similarity score between each query element in the query projection matrix and each cache element in the key-value cache of the target cache library; Based on the similarity score, the target cached data that matches the query request is determined from the key-value cache.

6. The method according to claim 5, wherein calculating the similarity score between each query element in the query projection matrix and each cache element in the key-value cache of the target cache library includes: Based on the target dimension set, the similarity sub-parameters between each query element in the query projection matrix and each cache element in the key-value cache of the target cache library are calculated respectively; wherein, the target dimension set includes at least a first dimension for characterizing semantic direction consistency, a second dimension for characterizing semantic matching strength, and a third dimension for characterizing semantic detail differences; The similarity sub-parameters are fused to obtain the similarity score.

7. The method according to claim 5, wherein determining the target cached data matching the query request from the key-value cache based on the similarity score comprises: Based on the distribution characteristics of the similarity scores, a dynamic matching threshold is determined; The cached elements whose similarity scores meet the dynamic matching threshold are identified as the target cached data; The step of outputting the target retrieval result corresponding to the query request based on the target cache data includes: Based on the similarity score, the attention weight corresponding to each target cached data is determined; The value cache matrix corresponding to the target cache data is weighted and aggregated based on the attention weights to generate the context representation of the query request; The context representation is input into the language model to generate the target retrieval result.

8. The method according to claim 1, further comprising: Based on the frequency of association between historical queries and semantic units, a target query association graph is constructed. In response to the query request, determine the historical queries that are semantically associated with the query request from the target query association graph; Based on the target semantic unit corresponding to the historical query, the key value of the target semantic unit is cached in the target storage area; The step of retrieving target cache data matching the query request from the target cache library includes: Retrieve the target cache data that matches the query request from the target storage area.

9. An information retrieval device, comprising: A receiving unit is used to receive a query request, wherein the query request represents a request to retrieve information from text content in a knowledge base; A matching unit is used to retrieve target cache data that matches the query request from the target cache library, wherein the target cache library is pre-built based on the text content in the knowledge base and is used to store key-value caches containing at least one semantic unit corresponding to the text content in the knowledge base, wherein the semantic unit is obtained by dividing the text content based on the semantic correlation between sentences in the text content; The output unit is used to output the target retrieval result corresponding to the query request based on the target cached data.

10. An electronic device comprising at least one processor and a target prediction model capable of running on the processor, the processor being configurable to perform the following operations: Receive a query request, which represents a request to retrieve information from text content in a knowledge base; Retrieve target cache data that matches the query request from the target cache library, wherein, The target cache library is a key-value cache pre-built based on the text content in the knowledge base, used to store at least one semantic unit corresponding to the text content in the knowledge base. The semantic unit is obtained by dividing the text content based on the semantic correlation between sentences in the text content. Based on the target cached data, output the target retrieval result corresponding to the query request.

Images