Search method, device and equipment

Abstract

The application discloses a retrieval method and device, and equipment. The method comprises the following steps: obtaining current question text input by a user; calculating the semantic similarity between the current question text and the stored questions in a preset cache by using a preset cross-encoder model, to obtain a similarity score corresponding to the current question text; obtaining the highest score in the similarity score as a target similarity score, and comparing the target similarity score with a preset threshold; if the target similarity score is greater than or equal to the preset threshold, obtaining the cache result associated with the stored question corresponding to the target similarity score from the preset cache, and generating a response output of the current question text based on the cache result; if the target similarity score is less than the preset threshold, or the preset cache is empty, obtaining a current response result based on retrieval enhancement, and taking the current response result as the response output of the current question text. The application improves the retrieval accuracy and efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a retrieval method, apparatus, and device. Background Technology

[0002] Retrieval-Augmented Generation (RAG) is a technical architecture that combines information retrieval with large language model generation. Its core idea is to first retrieve relevant document fragments from an external knowledge base when a user asks a question, and then input the retrieval results as context into a large language model, which then generates an answer.

[0003] Currently, retrieval methods typically process user questions using the following process: receiving the question text input by the user; encoding the question into a vector using an embedding model; performing vector similarity retrieval in a vector database (such as Milvus, Pinecone, etc.) to retrieve Top-K candidate document fragments; reordering the retrieved candidate documents according to their relevance to the question; concatenating the sorted Top-N document fragments into a context, inputting it into a large language model (such as GPT, Claude, etc.) to generate an answer; and finally returning the generated answer to the user.

[0004] However, the above methods have low retrieval efficiency. Summary of the Invention

[0005] The main purpose of this application is to provide a retrieval method, apparatus, or device that improves retrieval accuracy and efficiency.

[0006] To achieve the above objectives, firstly, this application provides a retrieval method, comprising: Get the current question text input by the user; The semantic similarity between the current question text and the questions stored in the preset cache is calculated using a preset cross-encoder model to obtain the similarity score corresponding to the current question text; The highest similarity score is obtained as the target similarity score, and the target similarity score is compared with a preset threshold. If the target similarity score is greater than or equal to the preset threshold, the cached result associated with the stored question corresponding to the target similarity score is retrieved from the preset cache, and the response output of the current question text is generated based on the cached result; If the target similarity score is less than the preset threshold, or the preset cache is empty, the current response result is obtained based on the search enhancement and is used as the response output for the current question text.

[0007] In one embodiment, a cross-encoder is used to calculate the semantic similarity between the current question text and questions already stored in a preset cache, to obtain a similarity score corresponding to the current question text, including: The current question text is combined with questions already stored in the preset cache to construct multiple question pairs; Multiple question pairs are input into a pre-defined cross-encoder model in batches, so that the pre-defined cross-encoder model performs joint self-attention encoding on the question text and the stored question in each of the multiple question pairs, generating an interactive semantic representation vector for each question pair. The cross-encoder model maps each interaction semantic representation vector to its classification head, and outputs the semantic similarity score for each question pair. The semantic similarity scores corresponding to each question are summarized to obtain the semantic similarity scores corresponding to multiple question pairs. The semantic similarity scores corresponding to multiple question pairs are then used as the similarity score corresponding to the current question text.

[0008] In one embodiment, retrieving cached results associated with stored questions corresponding to the target similarity score from a preset cache, and generating a response output of the current question text based on the cached results, includes: Extract the stored questions corresponding to the target similarity scores to obtain the matched cached question text; Extract cached results associated with the matched cached issue text, where the cached results include a sorted list of document fragments; After comparing the target similarity score with a preset threshold, a cache hit flag is generated; The cache hit flag, the matched cached question text, the target similarity score, and the cache result are associated to generate cache search results, which are then output as the response to the current question text.

[0009] In one embodiment, obtaining the current response result based on retrieval enhancement includes: The current question text is vector-encoded to obtain the question vector; Perform a similarity search on the question vector in the vector database to recall multiple candidate document fragments; The candidate document fragments from multiple candidate document fragments are cross-coded and reordered to obtain sorted candidate document fragments; The sorted candidate document fragments are input into a pre-defined large language model for inference to obtain the current response result.

[0010] In one embodiment, cross-coding reordering is performed on candidate document fragments from a plurality of candidate document fragments to obtain sorted candidate document fragments, including: The current question text is combined with each of the multiple candidate document fragments to construct multiple document-question pairs; The cross-encoder model sorts input documents by multiple document questions and scores their relevance to obtain a relevance score for each candidate document segment. The relevance scores corresponding to each candidate document fragment are summarized to obtain multiple candidate document fragments; The candidate document fragments are sorted in descending order of relevance score to obtain the sorted candidate document fragments.

[0011] In one embodiment, after obtaining the current response result based on retrieval enhancement, the method further includes: The current question text and the sorted list of document fragments in the current response result are associated and encapsulated to construct the current cache entry; Insert the current cache entry into the preset cache to obtain the incrementally updated preset cache.

[0012] In one embodiment, the method further includes: At least one of the following strategies is used to mark or evict cache entries: time-expiration eviction policy, capacity-limited eviction policy, and knowledge base update-triggered invalidation policy. The time-expiration eviction policy is used to characterize whether the lifespan of a cache entry exceeds a preset lifespan. When the lifespan exceeds the preset lifespan, the corresponding cache entry is marked as invalid. The capacity-limited eviction policy is used to characterize whether the number of cache entries exceeds the preset maximum capacity limit. When the number of cache entries exceeds the preset maximum capacity limit, the cache entries are evicted according to the least recently used policy. The knowledge base update triggering invalidation strategy is used to characterize whether there is an overlap between the document identifier affected by the knowledge base update event and the document identifier recorded in the cache entry. When there is an overlap, the corresponding cache entry is marked as invalid.

[0013] In one embodiment, the method further includes: Collect runtime statistics of the preset cache to obtain cache hit rate statistics and cache hit result accuracy score data; The cache hit rate statistics are compared with the preset hit rate lower limit. When the cache hit rate is lower than the preset hit rate lower limit, the first step of the long reduction operation is performed on the preset threshold. The accuracy score of the cache hit result is compared with the preset accuracy lower limit. When the accuracy score of the cache hit result is lower than the preset accuracy lower limit, a second step of length increase operation is performed on the preset threshold.

[0014] Secondly, embodiments of this application provide a retrieval device, including: The text acquisition module is used to acquire the current question text input by the user; The scoring calculation module is used to calculate the semantic similarity between the current question text and the questions stored in the preset cache using a preset cross-encoder model, and obtain the similarity score corresponding to the current question text. The comparison module is used to obtain the highest score among the similarity scores as the target similarity score, and compare the target similarity score with a preset threshold. The first output module is used to retrieve the cached result associated with the stored question corresponding to the target similarity score from the preset cache if the target similarity score is greater than or equal to the preset threshold, and generate the response output of the current question text based on the cached result; The second output module is used to obtain the current response result based on retrieval enhancement if the target similarity score is less than a preset threshold or the preset cache is empty, and output the current response result as the response to the current question text.

[0015] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0017] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0018] This application provides a retrieval method, apparatus, and device, including: acquiring a current question text input by a user; calculating the semantic similarity between the current question text and questions stored in a preset cache using a preset cross-encoder model to obtain a similarity score corresponding to the current question text; acquiring the highest score among the similarity scores as a target similarity score, and comparing the target similarity score with a preset threshold; if the target similarity score is greater than or equal to the preset threshold, acquiring cached results associated with stored questions corresponding to the target similarity score from the preset cache, and generating a response output for the current question text based on the cached results; if the target similarity score is less than the preset threshold, or the preset cache is empty, generating and acquiring a current response result based on retrieval enhancement, and using the current response result as the response output for the current question text. This application employs a cross-encoder model to score the semantic similarity between the current question and all cached questions. This model can identify questions that are essentially the same but presented in different ways, allowing semantically similar but not identical questions to also hit the cache. This effectively overcomes the technical deficiency of traditional exact string matching, which cannot identify semantically repetitive questions, thereby eliminating redundant computation while ensuring the quality of the answer. Furthermore, it automatically reverts to the complete RAG process when the cache is missed or empty, ensuring normal response capability to any new question. It also supports incremental updates to the cache after generating results, enabling the cached content to be continuously expanded and enriched. This constructs an efficient, accurate, and self-expanding question-answering retrieval optimization scheme. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 This is a flowchart illustrating a retrieval method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a retrieval device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0022] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0023] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0024] It should be understood that in this application, "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product or device.

[0025] It should be understood that in this application, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, "and / or B" can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0026] It should be understood that in this application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0027] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0028] The data involved in this application may be data authorized by the tester or fully authorized by all parties. The collection, dissemination, and use of the data shall comply with the relevant laws, regulations and standards of the relevant countries and regions. The implementation methods / executives of this application may be combined with each other.

[0029] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0030] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0031] Retrieval-Augmented Generation (RAG) is a technical architecture that combines information retrieval with large language model generation. Its core idea is to first retrieve relevant document fragments from an external knowledge base when a user asks a question, then input the retrieval results as context into a large language model, which then generates the answer. This technology effectively alleviates the illusion problem and knowledge update lag problem of large language models, and has become a mainstream technical solution for enterprise knowledge base question answering and intelligent customer service systems. Cross-encoders, as a model architecture that simultaneously inputs questions and documents into the same neural network model for joint encoding, can capture the deep semantic interaction between questions and documents. Their ranking accuracy is higher than that of dual-encoder methods based solely on vector similarity. Typical applications include using the Quora duplicate question detection model to determine whether two questions are semantically duplicated, and using the MS-MARCO ranking model to accurately re-rank retrieved documents. However, in real-world question answering scenarios with high concurrency and low latency requirements, how to reduce redundant computation and improve system response efficiency while ensuring answer quality remains a pressing technical problem to be solved.

[0032] Existing RAG question-answering systems typically process user questions using the following process: receiving the question text input by the user; encoding the question into a vector using an embedding model; performing vector similarity retrieval in a vector database (such as Milvus, Pinecone, etc.) to retrieve Top-K candidate document fragments; reordering the retrieved candidate documents using a cross-encoder, ranking them according to their relevance to the question; concatenating the ranked Top-N document fragments into a context, inputting it into a large language model (such as GPT, Claude, etc.) to generate an answer; and finally returning the generated answer to the user. Each step in this process consumes computational resources: vector retrieval requires querying the vector database and calculating similarity, cross-encoder reordering requires deep semantic scoring for each "question-document" pair, and large language model generation requires extensive forward inference computation. The entire process lacks a caching mechanism; each user question requires re-executing all steps from the beginning.

[0033] Existing technologies suffer from the following drawbacks: First, the system cannot identify semantically similar questions. It employs precise string matching or a no-caching strategy, and when users pose essentially the same question in different ways (e.g., "product warranty policy" and "how long is the warranty period"), the system cannot recognize their semantic equivalence, requiring the complete retrieval-sorting-generation process to be repeated. Second, in large-scale knowledge base scenarios, after vector retrieval recalls a large number of candidate documents, the cross-encoder needs to calculate a score for each candidate document. This process is computationally expensive and time-consuming, and for semantically repetitive questions, it is entirely redundant, resulting in slow response times. Third, the inference generation of large language models also consumes significant computational resources (GPU computing power, memory, etc.). For scenarios with already similar questions, repeated generation causes unnecessary resource consumption, especially in high-concurrency scenarios. Finally, existing solutions lack dynamic threshold adjustment and cache maintenance mechanisms, failing to automatically optimize cache hit rate and cache quality based on actual operating conditions, and unable to adapt to dynamic changes in knowledge base content, making it difficult to guarantee the timeliness and accuracy of cached results.

[0034] To address the aforementioned issues, this application provides a retrieval method.

[0035] Please see Figure 1 , Figure 1 This is a flowchart illustrating a retrieval method provided in an embodiment of this application. Figure 1 As shown, it includes the following steps: Step S101: Obtain the current question text input by the user.

[0036] Specifically, it receives the current question text entered by the user through the interactive interface. The question text can be any question in natural language form, such as "How do I apply for a refund?" or "What is the product warranty period?" entered by the user in the intelligent customer service.

[0037] Upon receiving the current question text, it is first preprocessed, including removing leading and trailing whitespace and standardizing the text encoding format to ensure data format consistency in subsequent processing. After preprocessing, the current question text is passed as input parameters for cache queries to the intelligent question caching module.

[0038] Step S102: Calculate the semantic similarity between the current question text and the questions stored in the preset cache using the preset cross-encoder model, and obtain the similarity score corresponding to the current question text.

[0039] In one embodiment, a cross-encoder is used to calculate the semantic similarity between the current question text and questions stored in a preset cache to obtain a similarity score corresponding to the current question text. This includes: combining the current question text with questions stored in the preset cache to construct multiple question pairs; inputting the multiple question pairs into a preset cross-encoder model in batches, so that the preset cross-encoder model performs joint self-attention encoding on the question text and stored questions in each of the multiple question pairs to generate an interaction semantic representation vector for each question pair; mapping each interaction semantic representation vector through the classification head of the cross-encoder model to output the semantic similarity score corresponding to each question pair; summarizing the semantic similarity scores corresponding to each question pair to obtain the semantic similarity scores corresponding to multiple question pairs, and using the semantic similarity scores corresponding to multiple question pairs as the similarity score corresponding to the current question text.

[0040] Specifically, the current question text is first combined with each question already stored in the preset cache to construct multiple question pairs. For example, if the cache already stores N historical questions, the current question is paired with each historical question to construct N question pairs. Then, these N question pairs are input into a preset cross-encoder model in batches. This cross-encoder model uses a joint encoding strategy, concatenating the two texts in each question pair into a single input sequence, in the format: [CLS] Sentence A [SEP]Sentence B [SEP]-1.

[0041] The concatenated sequence is input into a Transformer model (such as a cross-encoder / quora-distilroberta-base model based on the BERT architecture). The Transformer's self-attention mechanism models the cross-interaction relationship between the two texts in each encoding layer. The output vector at the [CLS] position is extracted as the joint feature representation of the question pair. Then, the cross-encoder model's classification head maps this interaction semantic representation vector, outputting a semantic similarity score for each question pair. This score is typically a floating-point number between 0 and 1; the closer the value is to 1, the more semantically similar the two questions are. The semantic similarity scores of all question pairs are summarized to obtain a similarity score list for the current question text.

[0042] Step S103: Obtain the highest score among the similarity scores as the target similarity score, and compare the target similarity score with a preset threshold.

[0043] Specifically, the highest score is selected from the similarity score list obtained in step S102 as the target similarity score, and this target similarity score is compared with a preset similarity threshold. The preset threshold can be 0.85, which is used to determine whether the current question is semantically similar enough to a stored question in the cache to constitute a cache hit. The higher the target similarity score, the greater the semantic overlap between the current question and the corresponding question in the cache.

[0044] While obtaining the target similarity score, the index or identifier of the corresponding cache issue is also recorded, so that the corresponding cached result can be quickly located and retrieved when a cache hit is determined. Step S104: If the target similarity score is greater than or equal to the preset threshold, retrieve the cached result associated with the stored question corresponding to the target similarity score from the preset cache, and generate the response output of the current question text based on the cached result.

[0045] In one embodiment, retrieving cached results associated with stored questions corresponding to the target similarity score from a preset cache, and generating a response output of the current question text based on the cached results, includes: extracting stored questions corresponding to the target similarity score to obtain matching cached question text; extracting cached results associated with the matching cached question text, wherein the cached results include a sorted list of document fragments; comparing the target similarity score with a preset threshold to generate a cache hit marker; associating the cache hit marker, the matching cached question text, the target similarity score, and the cached results to generate a cache lookup result, and using the cache lookup result as the response output of the current question text.

[0046] Specifically, if the target similarity score is greater than or equal to a preset threshold, it is determined to be a cache hit. The cached result associated with the stored question corresponding to the target similarity score is retrieved from the preset cache, and the response output of the current question text is generated based on the cached result.

[0047] Specifically, the process begins by extracting the stored questions corresponding to the target similarity score, yielding the matched cached question text (matched_query). Next, the cached results associated with this matched question text are extracted from the cache. These results include a sorted list of document fragments (i.e., the Top-N candidate document fragments obtained after vector retrieval and cross-encoder reordering of the historical question). The target similarity score is then compared with a preset threshold to generate a cache hit flag (hit=True). The cache hit flag, the matched cached question text, the target similarity score, and the cached results are then associated and encapsulated into a cache lookup result data structure (CacheLookupResult). This structure contains four fields: hit (a boolean value indicating whether a cache hit occurred), matched_query (the matched cached question text returned when a hit occurred), similarity_score (the similarity score with the best-matching cached question), and results (the cached search results returned when a hit occurred). This cache lookup result is then directly returned to the user as the response text for the current question, skipping all subsequent time-consuming steps such as vector retrieval, document reordering, and large language model generation.

[0048] Step S105: If the target similarity score is less than the preset threshold, or the preset cache is empty, then obtain the current response result based on the retrieval enhancement and output the current response result as the response of the current question text.

[0049] In one embodiment, obtaining the current response result based on retrieval enhancement includes: vector encoding the current question text to obtain a question vector; performing a similarity search on the question vector in a vector database to recall multiple candidate document fragments; cross-coding and reordering the candidate document fragments among the multiple candidate document fragments to obtain sorted candidate document fragments; and inputting the sorted candidate document fragments into a preset large language model for inference to obtain the current response result.

[0050] The process involves cross-coding and reordering candidate document fragments from multiple candidate document fragments to obtain sorted candidate document fragments. This includes: combining the current question text with each candidate document fragment from the multiple candidate document fragments to construct multiple document-question pairs; performing relevance scoring on the input document sorting cross-encoder model for the multiple document-question pairs to obtain a relevance score for each candidate document fragment; summing the relevance scores for each candidate document fragment to obtain multiple candidate document fragments; and sorting the multiple candidate document fragments in descending order of relevance score to obtain sorted candidate document fragments.

[0051] Specifically, if the target similarity score is less than a preset threshold, or the preset cache is empty, it is determined as a cache miss. Instead, the current response result is obtained based on the Retrieval Enhancement Generation (RAG) process, and this current response result is output as the response to the current question text. The specific implementation of the RAG process is as follows: First, the current question text is encoded into a question vector using an embedding model. This embedding model is typically a pre-trained dual-encoder model, capable of converting arbitrary text into a dense vector representation of fixed dimensions.

[0052] Secondly, the obtained question vector is subjected to similarity retrieval in a vector database (such as Milvus, Pinecone, etc.) to recall multiple candidate document fragments. The vector database uses the Approximate Nearest Neighbor (ANN) search algorithm to quickly retrieve the Top-K document fragments that are closest to the question vector in the vector space from a large-scale knowledge base as candidates.

[0053] Next, the recalled candidate document fragments are cross-encoded and reordered to obtain sorted candidate document fragments. Specifically, the current question text is combined with each of the multiple candidate document fragments to construct multiple document-question pairs. These document-question pairs are then input into a document-sorting cross-encoder model (e.g., cross-encoder / ms-marco-MiniLM-L6-v2) for relevance scoring. This model jointly encodes each "question-document" pair, capturing the deep semantic interaction between the question and the document through a self-attention mechanism, and outputs a relevance score for each candidate document fragment. The relevance scores for each candidate document fragment are summarized, and all candidate document fragments are sorted in descending order of relevance score to obtain sorted candidate document fragments. The cross-encoder reordering achieves higher accuracy than dual-encoder methods based solely on vector similarity, and can more accurately assess the relevance between documents and questions.

[0054] Finally, the sorted Top-N document fragments are concatenated into a context, which is then input into a pre-defined large language model (such as GPT, Claude, etc.) for inference. The large language model generates the current response result based on the retrieved context. This response result is the answer to the current question generated based on retrieval enhancement.

[0055] After the RAG process is completed, the current question text and its search results (including a sorted list of document fragments) are stored in the question cache as a new cache entry (CacheEntry) for reuse in subsequent queries of semantically similar questions. Each cache entry contains two fields: query (the original question text) and results (a list of sorted search results for the question, including RankedResult objects). Simultaneously, cache miss results are returned, including: hit flag hit=False, no matching question matched_query=None, similarity score similarity_score (i.e., the target similarity score obtained in step S103), and the newly generated search results results.

[0056] In one embodiment, after obtaining the current response result based on retrieval enhancement, the method further includes: The current question text and the sorted list of document fragments in the current response result are associated and encapsulated to construct the current cache entry; Insert the current cache entry into the preset cache to obtain the incrementally updated preset cache.

[0057] Specifically, the current question text and the sorted list of document fragments in the current response results are first associated and encapsulated to construct a new cache entry (CacheEntry). Each cache entry contains two core fields: the query field stores the original question text, and the results field stores the sorted search results list obtained after the question has undergone the complete RAG process. Each element in this list is a RankedResult object, containing information such as document identifier, document content, and relevance score. After encapsulation, the newly constructed cache entry is inserted into the preset cache. This insertion operation only adds a new entry to the existing cache set without affecting or rebuilding existing cache data, thus achieving incremental updates to the cache. After insertion, the new cache entry can immediately participate in semantic similarity matching queries for subsequent questions without waiting for cache rebuilding or restarting, ensuring the immediacy and efficiency of cache expansion.

[0058] In one embodiment, the method further includes: At least one of the following strategies is used to mark or evict cache entries: time-expiration eviction policy, capacity-limited eviction policy, and knowledge base update-triggered invalidation policy. The time-expiration eviction policy is used to characterize whether the lifespan of a cache entry exceeds a preset lifespan. When the lifespan exceeds the preset lifespan, the corresponding cache entry is marked as invalid. The capacity-limited eviction policy is used to characterize whether the number of cache entries exceeds the preset maximum capacity limit. When the number of cache entries exceeds the preset maximum capacity limit, the cache entries are evicted according to the least recently used policy. The knowledge base update triggering invalidation strategy is used to characterize whether there is an overlap between the document identifier affected by the knowledge base update event and the document identifier recorded in the cache entry. When there is an overlap, the corresponding cache entry is marked as invalid.

[0059] Specifically, this application employs at least one of the following strategies to mark or evict cache entries: time expiration eviction strategy, capacity limit eviction strategy, and knowledge base update triggered invalidation strategy. The three strategies can be used independently or in combination to form a multi-layered cache invalidation mechanism to ensure the timeliness and accuracy of cached results.

[0060] For the time-to-live (TTL) eviction policy, a preset TTL is set for each cached entry, with a default value of 24 hours. The creation timestamp of each cached entry is recorded upon creation, and the difference between the current time and the creation timestamp is checked during each cache query or background maintenance task to ensure it does not exceed the preset TTL. When the TTL exceeds the preset TTL, the corresponding cached entry is automatically marked as invalid. Caches marked as invalid will no longer participate in semantic similarity matching in subsequent cache queries and will be completely removed during periodic maintenance to prevent the return of expired results. The TTL value can be dynamically configured based on the knowledge base's update frequency. For frequently updated knowledge bases, a shorter TTL (e.g., 6 hours) can be set, while for relatively stable knowledge bases, a longer TTL (e.g., 48 hours) can be set.

[0061] For capacity-limited eviction policies, a maximum capacity limit (max_size) is set for the cache, representing the maximum number of entries the cache can hold. After each cache insertion operation, the total number of cache entries is checked to see if it exceeds the preset maximum capacity limit. When the number of cache entries exceeds the preset maximum capacity limit, the eviction mechanism is activated, and cache entries are evicted according to the Least Recently Used (LRU) policy. That is, cache entries that have not been hit for the longest time are removed first, while frequently hit hot entries are retained, thereby maximizing cache utilization efficiency within the limited cache space.

[0062] For the knowledge base update-triggered invalidation strategy, a list of document identifiers (document_ids) involved in the search results is recorded for each cached entry. This list is generated and stored based on the document sources in the search results when the cached entry is constructed. When an update event such as adding, modifying, or deleting a document occurs in the knowledge base, all document identifiers affected by the update event are obtained, and these document identifiers are compared with the document identifier list recorded in each cached entry. If an intersection exists, it means that the search results of that cached entry involve the updated document, and the corresponding cached entry is marked as invalid to ensure that the results returned by subsequent cache hits do not contain outdated knowledge content.

[0063] In one embodiment, the method further includes: Collect runtime statistics of the preset cache to obtain cache hit rate statistics and cache hit result accuracy score data; The cache hit rate statistics are compared with the preset hit rate lower limit. When the cache hit rate is lower than the preset hit rate lower limit, the first step of the long reduction operation is performed on the preset threshold. The accuracy score of the cache hit result is compared with the preset accuracy lower limit. When the accuracy score of the cache hit result is lower than the preset accuracy lower limit, a second step of length increase operation is performed on the preset threshold.

[0064] Specifically, two core statistical data sets for the preset cache are continuously recorded: cache hit rate statistics and cache hit result accuracy score data. The cache hit rate statistics are calculated as the proportion of cache hits to total cache queries within a preset time window. The cache hit result accuracy score data is obtained by periodically sampling and verifying cache hit results, comparing the results directly returned by the cache with the results obtained by re-executing the complete RAG process, evaluating the overlap of the Top-N document sets after sorting, and generating an accuracy score. Additionally, user satisfaction feedback on cache hit responses (such as likes) can be collected as a supplementary reference for the accuracy score.

[0065] After collecting the above statistical data, a dynamic threshold adjustment algorithm is executed. Specifically, the cache hit rate statistics are compared with a preset lower limit for the hit rate (default value is 30%). When the cache hit rate is lower than the preset lower limit, it indicates that the current threshold setting is too high, causing a large number of semantically similar questions to fail to hit the cache. The preset threshold is then lowered by a first step (0.05 each time) to reduce the hit threshold and improve the cache hit rate. Simultaneously, the cache hit result accuracy score data is compared with a preset lower limit for the accuracy rate (default value is 80%). When the cache hit result accuracy score is lower than the preset lower limit, it indicates that the current threshold setting is too low, causing some semantically incomplete questions to be misjudged as hits. The preset threshold is then increased by a second step (0.05 each time) to raise the hit threshold and ensure the accuracy of the cached results. The threshold adjustment frequency is once per hour to avoid excessively frequent adjustments that could lead to fluctuations and instability. In addition, constraints are set on the threshold adjustment range, limiting the threshold to the range of [0.70, 0.95] to prevent the threshold from being too low, resulting in a large number of false hits, or too high, resulting in almost no cache hits, thereby avoiding abnormal behavior caused by extreme thresholds.

[0066] This application provides a retrieval method, including: obtaining the current question text input by a user; calculating the semantic similarity between the current question text and questions stored in a preset cache using a preset cross-encoder model to obtain a similarity score corresponding to the current question text; obtaining the highest score among the similarity scores as the target similarity score, and comparing the target similarity score with a preset threshold; if the target similarity score is greater than or equal to the preset threshold, obtaining the cached results associated with the stored questions corresponding to the target similarity score from the preset cache, and generating a response output for the current question text based on the cached results; if the target similarity score is less than the preset threshold, or the preset cache is empty, obtaining the current response result based on retrieval enhancement, and using the current response result as the response output for the current question text. This application employs a cross-encoder model to score the semantic similarity between the current question and all cached questions. This model can identify questions that are essentially the same but presented in different ways, allowing semantically similar but not identical questions to also hit the cache. This effectively overcomes the technical deficiency of traditional exact string matching, which cannot identify semantically repetitive questions, thereby eliminating redundant computation while ensuring the quality of the answer. Furthermore, it automatically reverts to the complete RAG process when the cache is missed or empty, ensuring normal response capability to any new question. It also supports incremental updates to the cache after generating results, enabling the cached content to be continuously expanded and enriched. This constructs an efficient, accurate, and self-expanding question-answering retrieval optimization scheme.

[0067] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0068] The following are device embodiments of this application. For details not described in detail, please refer to the corresponding method embodiments described above.

[0069] Figure 2 This diagram illustrates the structure of a retrieval device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. The retrieval device includes: The text acquisition module 201 is used to acquire the current question text input by the user; The scoring calculation module 202 is used to calculate the semantic similarity between the current question text and the questions stored in the preset cache using a preset cross-encoder model, and to obtain the similarity score corresponding to the current question text. The comparison module 203 is used to obtain the highest score among the similarity scores as the target similarity score, and compare the target similarity score with a preset threshold. The first output module 204 is used to obtain the cached result associated with the stored question corresponding to the target similarity score from the preset cache if the target similarity score is greater than or equal to the preset threshold, and generate the response output of the current question text based on the cached result; The second output module 205 is used to obtain the current response result based on retrieval enhancement if the target similarity score is less than a preset threshold or the preset cache is empty, and output the current response result as the response of the current question text.

[0070] In one embodiment, the scoring calculation module 202 is further configured to combine the current question text with questions already stored in a preset cache to construct multiple question pairs; Multiple question pairs are input into a pre-defined cross-encoder model in batches, so that the pre-defined cross-encoder model performs joint self-attention encoding on the question text and the stored question in each of the multiple question pairs, generating an interactive semantic representation vector for each question pair. The cross-encoder model maps each interaction semantic representation vector to its classification head, and outputs the semantic similarity score for each question pair. The semantic similarity scores for each question pair are aggregated to obtain the semantic similarity scores for multiple question pairs. These multiple semantic similarity scores are then used as the similarity score for the current question text.

[0071] In one embodiment, the first output module 204 is further configured to extract the stored questions corresponding to the target similarity score to obtain the matched cached question text; Extract cached results associated with the matched cached issue text, where the cached results include a sorted list of document fragments; After comparing the target similarity score with a preset threshold, a cache hit flag is generated; The cache hit flag, the matched cached question text, the target similarity score, and the cache result are associated to generate cache search results, which are then output as the response to the current question text.

[0072] In one embodiment, the second output module 205 is further configured to perform vector encoding on the current question text to obtain a question vector; Perform a similarity search on the question vector in the vector database to recall multiple candidate document fragments; The candidate document fragments from multiple candidate document fragments are cross-coded and reordered to obtain sorted candidate document fragments; The sorted candidate document fragments are input into a pre-defined large language model for inference to obtain the current response result.

[0073] In one embodiment, the second output module 205 is further configured to combine the current question text with each of the multiple candidate document fragments to construct multiple document question pairs; The cross-encoder model sorts input documents by multiple document questions and scores their relevance to obtain a relevance score for each candidate document segment. The relevance scores corresponding to each candidate document fragment are summarized to obtain multiple candidate document fragments; The candidate document fragments are sorted in descending order of relevance score to obtain the sorted candidate document fragments.

[0074] In one embodiment, after the second output module 205, there is also an update module, which is used to associate and encapsulate the current question text and the sorted list of document fragments in the current response result to construct the current cache entry; Insert the current cache entry into the preset cache to obtain the incrementally updated preset cache.

[0075] In one embodiment, the apparatus further includes: a policy selection module, which is used to mark or evict cache entries using at least one of a time-expiration eviction policy, a capacity-limited eviction policy, and a knowledge base update-triggered invalidation policy, wherein... The time-expiration eviction policy is used to characterize whether the lifespan of a cache entry exceeds a preset lifespan. When the lifespan exceeds the preset lifespan, the corresponding cache entry is marked as invalid. The capacity-limited eviction policy is used to characterize whether the number of cache entries exceeds the preset maximum capacity limit. When the number of cache entries exceeds the preset maximum capacity limit, the cache entries are evicted according to the least recently used policy. The knowledge base update triggering invalidation strategy is used to characterize whether there is an overlap between the document identifier affected by the knowledge base update event and the document identifier recorded in the cache entry. When there is an overlap, the corresponding cache entry is marked as invalid.

[0076] In one embodiment, the device further includes a threshold adjustment module, which is used to collect running statistics data of the preset cache to obtain cache hit rate statistics data and cache hit result accuracy score data. The cache hit rate statistics are compared with the preset hit rate lower limit. When the cache hit rate is lower than the preset hit rate lower limit, the first step of the long reduction operation is performed on the preset threshold. The accuracy score of the cache hit result is compared with the preset accuracy lower limit. When the accuracy score of the cache hit result is lower than the preset accuracy lower limit, a second step of length increase operation is performed on the preset threshold.

[0077] This application provides a retrieval device, specifically used for: acquiring the current question text input by a user; calculating the semantic similarity between the current question text and questions stored in a preset cache using a preset cross-encoder model to obtain a similarity score corresponding to the current question text; acquiring the highest score among the similarity scores as the target similarity score, and comparing the target similarity score with a preset threshold; if the target similarity score is greater than or equal to the preset threshold, acquiring the cached results associated with the stored questions corresponding to the target similarity score from the preset cache, and generating a response output for the current question text based on the cached results; if the target similarity score is less than the preset threshold, or the preset cache is empty, generating and acquiring the current response result based on retrieval enhancement, and using the current response result as the response output for the current question text. This application employs a cross-encoder model to score the semantic similarity between the current question and all cached questions. This model can identify questions that are essentially the same but presented in different ways, allowing semantically similar but not identical questions to also hit the cache. This effectively overcomes the technical deficiency of traditional exact string matching, which cannot identify semantically repetitive questions, thereby eliminating redundant computation while ensuring the quality of the answer. Furthermore, it automatically reverts to the complete RAG process when the cache is missed or empty, ensuring normal response capability to any new question. It also supports incremental updates to the cache after generating results, enabling the cached content to be continuously expanded and enriched. This constructs an efficient, accurate, and self-expanding question-answering retrieval optimization scheme.

[0078] This application Figure 3 A schematic diagram of a computer device is provided. (Example) Figure 3 As shown, the computer device 3 in this embodiment includes a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various retrieval method embodiments described above, for example... Figure 1 Steps 101 to 105 are shown. Alternatively, when processor 301 executes computer program 303, it implements the functions of each module / unit in the above-described retrieval device embodiments, for example... Figure 2 The functions of modules / units 201 to 205 are shown.

[0079] This application also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the retrieval methods provided in the various embodiments described above.

[0080] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0081] This application also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the execution of the executable instructions by the at least one processor causes the device to implement the retrieval methods provided in the various embodiments described above.

[0082] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0083] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A retrieval method, characterized in that, include: Get the current question text input by the user; The semantic similarity between the current question text and the questions stored in the preset cache is calculated using a preset cross-encoder model to obtain the similarity score corresponding to the current question text; The highest score among the similarity scores is obtained as the target similarity score, and the target similarity score is compared with a preset threshold. If the target similarity score is greater than or equal to the preset threshold, then the cached result associated with the stored question corresponding to the target similarity score is obtained from the preset cache, and the response output of the current question text is generated based on the cached result; If the target similarity score is less than the preset threshold, or the preset cache is empty, the current response result is obtained based on retrieval enhancement and the current response result is used as the response output for the current question text.

2. The retrieval method as described in claim 1, characterized in that, The step of calculating the semantic similarity between the current question text and questions already stored in a preset cache using a cross-encoder to obtain a similarity score corresponding to the current question text includes: The current question text is combined with questions already stored in the preset cache to construct multiple question pairs; The multiple question pairs are input into a preset cross-encoder model in batch form, so that the preset cross-encoder model performs joint self-attention encoding on the question text and the stored question in each of the multiple question pairs, generating an interactive semantic representation vector for each question pair. The cross-encoder model maps each interaction semantic representation vector to its classification head, and outputs the semantic similarity score corresponding to each question pair. The semantic similarity scores corresponding to each question pair are summarized to obtain the semantic similarity scores corresponding to the multiple question pairs, and the semantic similarity scores corresponding to the multiple question pairs are used as the similarity score corresponding to the current question text.

3. The retrieval method as described in claim 1, characterized in that, The step of retrieving cached results associated with stored questions corresponding to the target similarity score from the preset cache, and generating a response output of the current question text based on the cached results, includes: Extract the stored questions corresponding to the target similarity scores to obtain the matched cached question text; Extract the cache results associated with the matched cache issue text, wherein the cache results include a sorted list of document fragments; After comparing the target similarity score with a preset threshold, a cache hit flag is generated; The cache hit marker, the matched cached question text, the target similarity score, and the cache result are associated to generate a cache lookup result, and the cache lookup result is output as the response to the current question text.

4. The retrieval method as described in claim 1, characterized in that, The process of generating and obtaining the current response result based on search enhancement includes: The current question text is vector-encoded to obtain the question vector; A similarity search is performed on the question vector in the vector database to recall multiple candidate document fragments; The candidate document fragments among the plurality of candidate document fragments are cross-coded and reordered to obtain sorted candidate document fragments; The sorted candidate document fragments are input into a preset large language model for inference to obtain the current response result.

5. The retrieval method as described in claim 4, characterized in that, The step of cross-coding and reordering the candidate document fragments from the plurality of candidate document fragments to obtain sorted candidate document fragments includes: The current question text is combined with each of the multiple candidate document fragments to construct multiple document question pairs; The relevance score of each candidate document segment is obtained by performing a relevance score on the input document sorting cross-encoder model based on the multiple document questions. The relevance scores corresponding to each candidate document fragment are summarized to obtain multiple candidate document fragments; The candidate document fragments are sorted in descending order of their relevance scores to obtain the sorted candidate document fragments.

6. The retrieval method as described in claim 5, characterized in that, After obtaining the current response result based on search enhancement, the process also includes: The current question text and the sorted list of document fragments in the current response result are associated and encapsulated to construct the current cache entry; The current cache entry is inserted into the preset cache to obtain the incrementally updated preset cache.

7. The retrieval method as described in claim 1, characterized in that, The method further includes: At least one of the following strategies is used to mark or evict cache entries: time-expiration eviction policy, capacity-limited eviction policy, and knowledge base update-triggered invalidation policy. The time expiration eviction policy is used to characterize whether the lifespan of a cache entry exceeds a preset lifespan. When the lifespan exceeds the preset lifespan, the corresponding cache entry is marked as invalid. The capacity-limited eviction policy is used to characterize whether the number of cache entries exceeds the preset maximum capacity limit. When the number of cache entries exceeds the preset maximum capacity limit, the cache entries are evicted according to the least recently used policy. The knowledge base update triggering invalidation strategy is used to characterize whether there is an overlap between the document identifier affected by the knowledge base update event and the document identifier recorded in the cache entry. When there is an overlap, the corresponding cache entry is marked as invalid.

8. The retrieval method as described in claim 1, characterized in that, The method further includes: The running statistics of the preset cache are collected to obtain cache hit rate statistics and cache hit result accuracy score data; The cache hit rate statistics are compared with a preset lower limit. When the cache hit rate is lower than the preset lower limit, the first step of the length reduction operation is performed on the preset threshold. The accuracy score of the cache hit result is compared with a preset accuracy lower limit. When the accuracy score of the cache hit result is lower than the preset accuracy lower limit, a second step of length increase operation is performed on the preset threshold.

9. A retrieval device, characterized in that, include: The text acquisition module is used to acquire the current question text input by the user; The scoring calculation module is used to calculate the semantic similarity between the current question text and the questions stored in the preset cache using a preset cross-encoder model, and obtain the similarity score corresponding to the current question text. The comparison module is used to obtain the highest score among the similarity scores as the target similarity score, and compare the target similarity score with a preset threshold; The first output module is used to obtain the cached result associated with the stored question corresponding to the target similarity score from the preset cache if the target similarity score is greater than or equal to the preset threshold, and generate the response output of the current question text based on the cached result; The second output module is used to obtain the current response result based on retrieval enhancement if the target similarity score is less than the preset threshold or the preset cache is empty, and to output the current response result as the response to the current question text.

10. A computer device, characterized in that, Includes a memory, and one or more processors communicatively connected to the memory; The memory stores instructions that can be executed by the one or more processors to cause the one or more processors to implement the retrieval method as described in any one of claims 1 to 8.

Images