Retrieval enhancement generation method and device and related equipment

Abstract

The invention provides a retrieval enhancement generation method and device and related equipment, and relates to the technical field of natural language processing. The method comprises the following steps: acquiring a query corpus; inputting the query corpus into a retriever, and outputting a first retrieval result; inputting the first retrieval result into a pre-trained retrieval optimization generation model, and outputting a second retrieval result; wherein the retrieval optimization generation model is obtained by training according to a historical query corpus and a query result corresponding to the historical query corpus, and the query result corresponding to the historical query corpus is obtained by outputting a large language model; and inputting the second retrieval result into the large language model, and outputting a query result. According to the method, the quality of the retrieval result input into the large language model can be improved, so that the answer generated by the large language model is more accurate and more coherent in logic, the generalization ability of the model is enhanced, and the method can better adapt to diversified application scenes.

Description

Technical Field

[0001] The present disclosure relates to the field of natural language processing technology, and in particular to a retrieval enhancement generation method, apparatus, and related equipment. Background Art

[0002] Retrieval-Augmented Generation (RAG) is an AI solution that combines retrieval and generation capabilities. Its core is to use a retriever to quickly find document fragments relevant to user questions from an external knowledge base. These document fragments are then fed into a large language model (LLM) as context. The LLM combines the retrieved precise information with its own training knowledge to generate more accurate and higher-quality answers.

[0003] However, the retriever and large language model in RAG technology are usually trained independently, resulting in a "preference gap" between the two in task processing. Among them, the retriever is designed based on human preferences, with the goal of efficiently screening documents related to the query, but its evaluation criteria focus on surface relevance and may return redundant or low-quality content. LLM relies on deep semantic understanding to generate answers and is sensitive to noise. Too much irrelevant information will interfere with its reasoning, resulting in inaccurate output. In addition, there are significant differences in the focus of the two in processing queries. The retriever tends to quickly match keywords, while the LLM requires coherent and precise semantic associations. This misalignment of goals and capabilities makes it easy for information bias or logical breaks to occur when the two collaborate, ultimately affecting the overall effectiveness of the question-answering system.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0005] The present disclosure provides a retrieval enhancement generation method, apparatus, and related equipment, which can make the generated answers more accurate and logically coherent, enhance the generalization ability of the model, and better adapt to diverse application scenarios.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a retrieval enhancement generation method is provided, the method comprising: obtaining a query corpus; inputting the query corpus into a retriever and outputting a first retrieval result; inputting the first retrieval result into a retrieval optimization generation model and outputting a second retrieval result; wherein the retrieval optimization generation model is trained based on historical query corpus and query results corresponding to the historical query corpus, and the query results corresponding to the historical query corpus are output by a large language model; inputting the second retrieval result into the large language model and outputting a query result.

[0008] In some embodiments, inputting the first search result into a search optimization generation model and outputting a second search result includes: assigning a unique identifier to each search statement in the first search result to obtain a first search statement set; calculating the similarity between each search statement in the first search statement set and the query corpus to obtain a relevance score between each search statement and the query corpus; selecting search statements whose relevance scores are greater than a preset threshold to obtain a second search statement set; and splicing the second search statement set with the query corpus to obtain a second search result.

[0009] In some embodiments, the step of splicing the second search statement set with the query corpus to obtain a second search result further includes: inputting the second search result into a retriever to output a third search result; inputting the third search result into the retrieval optimization generation model to assign a unique identifier to each search statement in the third search result to obtain a third search statement set; calculating the similarity between each search statement in the third search statement set and the query corpus to obtain a relevance score between each search statement and the query corpus; selecting search statements with a relevance score greater than a preset threshold to obtain a fourth search statement set; splicing the fourth search statement set with the query corpus to obtain a fourth search result; the step of inputting the second search result into the large language model to output a query result includes: inputting the fourth search result into the large language model to output a query result.

[0010] In some embodiments, the retrieval optimization generative model is trained by:

[0011] At least one training sample is obtained, each training sample includes a query corpus sample and a label preference annotation result corresponding to the query corpus sample, and the label preference annotation result is a comprehensive annotation result obtained by performing preference annotation on the query result output by the large language model and the retrieval result output by the retriever; a predicted query result corresponding to the query corpus sample is generated according to the query corpus sample through a retrieval optimization generation model; the retrieval optimization generation model is trained according to the label preference annotation result and the predicted query result to obtain a trained retrieval optimization generation model, and the trained retrieval optimization generation model is used to optimize the retrieval result output by the retriever.

[0012] In some embodiments, obtaining at least one training sample includes: obtaining at least one sample query corpus, each sample query corpus having a corresponding sample text; determining the label preference annotation results corresponding to the sample text based on the query results output by the large language model corresponding to the sample query corpus and the retrieval results output by the retriever; obtaining the training sample based on the sample text corresponding to the sample query corpus and the label preference annotation results corresponding to the sample text.

[0013] In some embodiments, determining the label preference annotation result corresponding to the sample text based on the query result output by the large language model corresponding to the sample query corpus and the retrieval result output by the retriever includes: determining the preference information of the query result output by the large language model based on the correctness of the query result output by the large language model corresponding to the sample query corpus; determining the preference information of the retrieval result output by the retriever based on the correctness of the query result output by the large language model associated with the retrieval result output by the retriever; determining the comprehensive annotation result of the sample query corpus based on the preference information of the query result output by the large language model and the preference information of the retrieval result output by the retriever; and determining the comprehensive annotation result corresponding to the sample text based on the correspondence between the sample query corpus and the sample text and the comprehensive annotation result of the sample query corpus.

[0014] In some embodiments, the retrieval optimization generation model is trained according to the label preference annotation results and the query results to obtain the trained retrieval optimization generation model, including: determining a first loss function value according to the label preference annotation results and the predicted query results; and training the retrieval optimization generation model according to the first loss function value to obtain the trained retrieval optimization generation model.

[0015] In some embodiments, the method further includes: in an offline stage, encoding the document in the retriever based on a Transformer encoder to obtain an embedding vector of the document; storing the embedding vector of the document in a vector retrieval library; and pre-filling the document with a large language model, calculating the key-value matrix of the document, and caching it.

[0016] In some embodiments, inputting the query corpus into the retriever and outputting the first retrieval result includes: encoding the query corpus to obtain a query embedding vector; and searching the vector retrieval library for a target embedding vector whose similarity is greater than a preset threshold based on the query embedding vector to obtain the first retrieval result.

[0017] In some embodiments, inputting the second retrieval result into the large language model and outputting the query result includes: using the large language model to determine the key-value matrix of the second retrieval result based on the second retrieval result; inputting the key-value matrix of the second retrieval result as a pre-filled context into the large language model to generate a pre-filled result; generating a new token sequence based on the pre-filled result by decoding; and outputting the query result according to the new token sequence.

[0018] In some embodiments, the key-value matrix of the second retrieval result includes key-value matrices corresponding to multiple documents, and the step of inputting the key-value matrix of the second retrieval result as a pre-filled context into the large language model to generate a pre-filled result includes: merging the key-value matrices corresponding to the multiple documents to generate an initial context, wherein the initial context includes the original composite location identifier of each document; performing a remapping operation on the original composite location identifier of each document to replace the original composite location identifier of each document with a rearranged location identifier; updating location-related information in the key state based on the remapped location identifier; and inputting the updated key-value cache as a pre-filled context into the large language model to generate a pre-filled result.

[0019] In some embodiments, inputting the first retrieval result into a retrieval optimization generation model and outputting a second retrieval result includes: inputting the first retrieval result into a retrieval optimization generation model, determining a key-value matrix of the first retrieval result based on the first retrieval result; calculating an enhancement weight of the key-value matrix of the first retrieval result using the retrieval optimization generation model; screening document paragraphs in the first retrieval result according to the enhancement weight to obtain a document subset; performing context enhancement on the document subset, and outputting a second retrieval result.

[0020] According to another aspect of the present disclosure, a retrieval enhancement generation device is also provided, which includes: an acquisition module for acquiring a query corpus; a first retrieval module for inputting the query corpus into a retriever and outputting a first retrieval result; a second retrieval module for inputting the first retrieval result into a retrieval optimization generation model and outputting a second retrieval result; wherein the retrieval optimization generation model is trained based on historical query corpus and query results corresponding to the historical query corpus, and the query results corresponding to the historical query corpus are output by a large language model; a query module for inputting the second retrieval result into the large language model and outputting a query result.

[0021] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned retrieval enhancement generation methods by executing the executable instructions.

[0022] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements any one of the above-mentioned search enhancement generation methods.

[0023] According to another aspect of the present disclosure, a computer program product is provided, including: a computer program or instructions, wherein when the computer program or instructions are executed by a processor, any one of the above-mentioned search enhancement generation methods is implemented.

[0024] A retrieval enhancement generation method, apparatus, and related equipment provided in an embodiment of the present disclosure include: obtaining a query corpus; inputting the query corpus into a retriever and outputting a first retrieval result; inputting the first retrieval result into a pre-trained retrieval optimization generation model and outputting a second retrieval result; wherein the retrieval optimization generation model is trained based on historical query corpus and query results corresponding to the historical query corpus, and the query results corresponding to the historical query corpus are output by a large language model; inputting the second retrieval result into the large language model and outputting a query result. The retrieval optimization generation model in this embodiment is trained based on historical query corpus and query results corresponding to the historical query corpus, and the second retrieval result output by the retrieval optimization generation model can effectively align with the output selection tendency of the large language model, thereby improving the quality of the retrieval results input into the large language model, making the answers generated by the large language model more accurate and logically coherent, enhancing the model's generalization ability, and being better able to adapt to diverse application scenarios.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0027] Figure 1 A schematic diagram showing the system structure of a search enhancement generation method according to an embodiment of the present disclosure is shown;

[0028] Figure 2 A flow chart of a search enhancement generation method according to an embodiment of the present disclosure is shown;

[0029] Figure 3 A flow chart of a method for outputting a second search result in an embodiment of the present disclosure is shown;

[0030] Figure 4 A flowchart of another search enhancement generation method according to an embodiment of the present disclosure is shown;

[0031] Figure 5 A flow chart of a method for training a retrieval optimization generation model according to an embodiment of the present disclosure is shown;

[0032] Figure 6 A flow chart of a method for obtaining training samples according to an embodiment of the present disclosure is shown;

[0033] Figure 7 A flowchart of another search enhancement generation method according to an embodiment of the present disclosure is shown;

[0034] Figure 8A A flow chart of a method for outputting a first search result in an embodiment of the present disclosure is shown;

[0035] Figure 8B A flow chart of a method for outputting query results in an embodiment of the present disclosure is shown;

[0036] Figure 9 A flow chart of a method for outputting a second search result in an embodiment of the present disclosure is shown;

[0037] Figure 10 A flowchart of another method for search enhancement generation according to an embodiment of the present disclosure is shown;

[0038] Figure 11 A flowchart of another method for search enhancement generation according to an embodiment of the present disclosure is shown;

[0039] Figure 12 A flowchart of another method for search enhancement generation according to an embodiment of the present disclosure is shown;

[0040] Figure 13 A flowchart of another method for search enhancement generation according to an embodiment of the present disclosure is shown;

[0041] Figure 14 A schematic diagram of a search enhancement generation device according to an embodiment of the present disclosure is shown;

[0042] Figure 15 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0043] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0044] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0045] For ease of understanding, before introducing the embodiments of the present disclosure, several terms involved in the embodiments of the present disclosure are first explained as follows:

[0046] Time to First Token (TTFT): This refers to the time it takes for a Retrieval-Augmented Generation (RAG) system or large language model to generate the first valid output (i.e., the first token) after receiving user input. It is a key metric for measuring model responsiveness and directly impacts user experience.

[0047] Large Language Model (LLM): A deep learning model trained on massive amounts of text data that can understand and generate human language. Its core is the Transformer architecture, which uses a self-attention mechanism to capture relationships between words and achieve contextual understanding.

[0048] A pre-trained model (PTM), also known as a base model or large model, refers to a deep neural network (DNN) with a large number of parameters. It is trained on massive amounts of unlabeled data. Leveraging the function approximation capabilities of large-parameter DNNs, the PTM extracts common features from the data. Through fine-tuning and efficient parameter fine-tuning (including prompt tuning, prefix tuning, adapters, and LoRA), the PTM is then adapted for downstream tasks. Therefore, pre-trained models can achieve ideal results in few-shot or zero-shot scenarios. PTMs can be categorized into language models, vision models, speech models, and multimodal models based on the data modality they process. Multimodal models represent features from two or more data modalities. Pre-trained models are important tools for outputting AI-generated content and can also serve as a universal interface for connecting multiple task-specific models. The pre-trained model in the embodiments of the present disclosure can be considered a pre-trained model.

[0049] A key-value (KV) cache is an efficient data storage structure that stores data in a "key-value" format. The key is a unique identifier used for fast lookup; the value is the associated data content.

[0050] Transformer encoder: It is one of the core components of the Transformer architecture, used to process input sequences and extract high-level features. Its structure consists of two sub-layers: a multi-head self-attention mechanism and a feedforward neural network, each followed by a residual connection and layer normalization. Among them, the multi-head self-attention mechanism calculates the relationship weights of all positions in the input sequence in parallel, capturing global dependencies and avoiding cyclic structures. The feedforward neural network performs nonlinear transformations independently on each position to enhance expressive power. The encoder gradually abstracts features by stacking multiple layers (such as 12 layers of BERT), and finally outputs context-related sequence representations for use in downstream tasks (such as classification and translation).

[0051] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0052] Figure 1 FIG. 1 shows an exemplary application system architecture diagram to which the search enhancement generation method in the embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture may include a terminal device 101 , a network 102 and a server 103 .

[0053] The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103 , and can be a wired network or a wireless network.

[0054] Optionally, the above-mentioned wireless network or wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0055] The terminal device 101 can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0056] Optionally, the client of the application installed in different terminal devices 101 is the same, or the client of the same type of application based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile phone client, a PC client, etc.

[0057] The server 103 may be a server that provides various services, such as a background management server that provides support for the devices operated by the user using the terminal device 101. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device.

[0058] Optionally, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0059] Those skilled in the art will know that Figure 1 The number of terminal devices, networks, and servers in the embodiment is merely illustrative, and any number of terminal devices, networks, and servers may be provided based on actual needs. This embodiment of the present disclosure does not limit this.

[0060] Under the above system architecture, an embodiment of the present disclosure provides a retrieval enhancement generation method, which can be executed by any electronic device with computing and processing capabilities.

[0061] In some embodiments, the retrieval enhancement generation method provided in the embodiments of the present disclosure can be executed by a terminal device of the above-mentioned system architecture; in other embodiments, the retrieval enhancement generation method provided in the embodiments of the present disclosure can be executed by a server in the above-mentioned system architecture; in other embodiments, the retrieval enhancement generation method provided in the embodiments of the present disclosure can be implemented by the terminal device and server in the above-mentioned system architecture through interaction.

[0062] Figure 2 A flowchart of a search enhancement generation method according to an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the search enhancement generation method provided in the embodiment of the present disclosure includes the following steps:

[0063] S202: Obtain query corpus.

[0064] In this embodiment, the query corpus refers to the original text collection used for tasks such as information retrieval and language analysis. It contains data such as sentences and paragraphs required for user queries or system analysis. It should be noted that the query corpus comes in various forms, including but not limited to text, and can also include voice, images, videos, and even sensor data (such as voice assistant recordings and images for image search). The essence of the query corpus is to provide semantic information through raw data to support the system in completing understanding, matching, or analysis tasks.

[0065] For example, in information retrieval, query corpus usually refers to keywords or questions entered by users; in natural language processing, query corpus can refer to text datasets used to train models (such as question-answer pairs, news articles, etc.).

[0066] S204: Input the query corpus into the search engine and output the first search result.

[0067] In this embodiment, the retriever is the core component of the RAG system. It uses vector similarity matching (such as the FAISS algorithm) to quickly find the most relevant information from the knowledge base to the query corpus. It serves as the knowledge input pipeline for the generator (large language model), ensuring that the model generates content based on real context and reducing the "hallucination" problem of the large language model. The first search result, which is the search result output by the retriever after sorting by similarity, is usually the content fragment with the highest semantic relevance to the query in the knowledge base and can serve as a reference for the generator.

[0068] For example, when a user enters "quantum computing principles," the search engine converts the query into a vector, compares it with the vectors of documents in the knowledge base, and sorts them by similarity to output the top-k document paragraphs, which are the k most semantically related pieces of knowledge, such as those on quantum bits and superposition states. These top-k document paragraphs serve as input to the large language model, providing it with real-world context and improving the accuracy of its output.

[0069] S206, input the first search result into the search optimization generation model, and output the second search result; wherein, the search optimization generation model is trained based on the historical query corpus and the query results corresponding to the historical query corpus, and the query results corresponding to the historical query corpus are output by the large language model.

[0070] In this embodiment, the retrieval optimization generation model is trained based on historical query data and the corresponding LLM generation results. It can learn the correlation patterns between high-quality search results and query results. Specifically, after the first search result is input into the retrieval optimization generation model, the retrieval optimization generation model will filter and / or enhance the document (such as removing noise and supplementing key information) and output a second search result that is more suitable for the LLM.

[0071] In other words, the retrieval optimization generation model serves as an "intermediate shim model" between the retriever and the LLM in the RAG system. A trained retrieval optimization generation model can coordinate the preference differences between the retriever and the LLM, and optimize the content input to the LLM by screening and enhancing the document information corresponding to the first search result retrieved, thereby improving the overall performance of the RAG system. It should be noted that the "intermediate shim model" between the retriever and the LLM in the RAG system has no fixed name. "Shim" emphasizes its connecting role, and "optimization generation" points to the functional goal. In actual applications, the name can be customized, such as adapter, enhancement model, etc. This embodiment does not limit the specific naming method.

[0072] S208: Input the second search result into the large language model and output the query result.

[0073] In this embodiment, the document paragraph (e.g., text, table, etc.) corresponding to the second search result is used as input data and fed into the large language model. The large language model analyzes, summarizes, or answers preset questions based on the input content to generate a final response. It is understood that in this embodiment, the second search result fed into the large language model has been optimized by the search optimization generation model, allowing the LLM to more accurately extract information. The two work together to form an efficient "search → optimization → generation" chain in the RAG system, improving the accuracy of answers.

[0074] In the above embodiment, the retrieval optimization generation model is trained based on the historical query corpus and the query results corresponding to the historical query corpus. The second retrieval result output by the retrieval optimization generation model can effectively align with the output selection tendency of the large language model, thereby improving the quality of the retrieval results input into the large language model, making the answers generated by the large language model more accurate and logically more coherent, enhancing the model's generalization ability, and better adapting to diverse application scenarios.

[0075] In some embodiments, Figure 3 A flow chart of a method for outputting a second search result provided by an embodiment of the present disclosure, combined with Figure 3 As shown, the present embodiment provides that inputting the first search result into the search optimization generation model and outputting the second search result may include the following steps:

[0076] S302: Assign a unique identifier to each search statement in the first search result to obtain a first search statement set.

[0077] In this embodiment, the first search result refers to the initial result list obtained by the retriever, which usually contains multiple related statements or records. The search statement is each specific content in the first search result, such as the text fragment, question or keyword combination returned by the search. Assigning a unique identifier means assigning a unique ID (such as a digital number, hash value, etc.) to each search statement to distinguish and track the corresponding search statement. The first search statement set is a structured set of all identified search statements to facilitate subsequent management, reference or analysis.

[0078] For example, for each document paragraph P returned by the retriever, k}, the retrieval optimization generation model is for each sentence P1, P2, ..., P k Assign a unique sentence ID (SID) s1, s2, ..., s n , forming a SID set S={s1,s2,…,s n}.

[0079] S304: Calculate the similarity between each search statement in the first search statement set and the query corpus to obtain a relevance score between each search statement and the query corpus.

[0080] In this embodiment, based on the input query corpus Q, the retrieval optimization generation model calculates each retrieval sentence s in the first retrieval sentence set. j The similarity with the query corpus Q, and the relevance score r of each search statement and the query corpus is obtained j .

[0081] It should be noted that there are many possible implementation methods for calculating similarity, including but not limited to cosine similarity, Jaccard similarity, and Euclidean distance. Taking cosine similarity calculation as an example, the formula is as follows:

[0082]

[0083] Among them, q is the vector representation of query Q, s j is the vector representation of the jth search sentence in the first search sentence set.

[0084] S306: Select search statements with a relevance score greater than a preset threshold to obtain a second search statement set.

[0085] The preset threshold is a pre-set numerical standard (such as 0.7) used to determine whether the correlation meets the standard.

[0086] S308: Concatenate the second search statement set with the query corpus to obtain a second search result.

[0087] In this embodiment, the original query Q is enhanced using the sentence corresponding to the search statement ID in the screened second search statement set to obtain an enhanced query, ie, the second search result.

[0088] q′=q+∑s j ∈S′s j

[0089] Among them, q is the vector representation of query Q, s j is the vector representation of the jth search sentence in the first search sentence set.

[0090] Specifically, each document fragment or sentence in the second search statement set and the query corpus is encoded into an embedding vector, and the embedding vector sequence of the second search statement set and the query corpus constitutes the input of the retrieval optimization generation model.

[0091] It should be noted that the core of the retrieval-optimized generative model is a trainable neural architecture (e.g., Transformer). The attention layers within the retrieval-optimized generative model (specifically, self-attention or encoder-decoder attention) allow each segment vector in the embedding vector sequence corresponding to the second set of search statements to interact with all other segment vectors in the embedding vector sequence and the query vector corresponding to the query corpus.

[0092] For each segment in the sequence, the attention mechanism computes a score indicating the segment's importance or relevance relative to other segments in the sequence. These scores determine which segments should be "focused" on and to what extent when generating the fused representation. Based on the computed attention scores, the retrieval-optimized generative model generates a weighted context vector to represent each segment. This vector is not a simple copy of the original segment, but rather a fused representation that incorporates all relevant information from the entire input sequence.

[0093] Furthermore, the fused vector output by the attention layer will pass through one or more linear layers (which may include activation functions and nonlinear transformations). The fused vector will further refine the information after passing through one or more linear layers, while learning more complex feature combinations and mapping the fused high-dimensional representation to the final desired output space.

[0094] In this embodiment, the retrieval optimization generation model can coordinate the selection tendencies of the retriever and LLM, optimize the content input to the LLM by screening and enhancing the retrieved document information, and thus improve the overall performance of the RAG system.

[0095] In some embodiments, Figure 4 This is a flow chart of a search enhancement generation method according to an embodiment of the present disclosure, Figure 4 As shown, this embodiment concatenates the second search statement set with the query corpus to obtain a second search result, and further includes the following steps:

[0096] S402: Input the second search result into the retriever, and output the third search result.

[0097] In this embodiment, the retriever performs a secondary search based on the enhanced query q′ to obtain a third search result, ie, a new top-k′ document paragraph set P′.

[0098] S404: Input the third search result into the search optimization generation model, assign a unique identifier to each search statement in the third search result, and obtain a third search statement set.

[0099] S406: Calculate the similarity between each search statement in the third search statement set and the query corpus to obtain a relevance score between each search statement and the query corpus.

[0100] S408 : Select search statements with a relevance score greater than a preset threshold to obtain a fourth search statement set.

[0101] S410: Concatenate the fourth search statement set with the query corpus to obtain a fourth search result.

[0102] In this embodiment, the retrieval optimization generation model again screens the new top-k' document paragraph set P', and finally obtains the sentence set S that is input to the LLM final .

[0103] S412: Input the fourth search result into the large language model and output the query result.

[0104] In this embodiment, the retrieval optimization generation model assigns unique IDs to sentences in documents returned by the search engine, calculates the sentence's relevance score to the query, selects relevant sentences to enhance the query, and then, after a secondary search and final screening, feeds the optimized content into the LLM. In this way, by coordinating the selection preferences of the search engine and the LLM, the retrieval optimization generation model accurately filters and enhances search information, providing high-quality input to the LLM. This makes the generated answers more accurate and logically coherent, enhances the model's generalization capabilities, and better adapts to diverse application scenarios.

[0105] In some embodiments, Figure 5 A flow chart of a method for training a retrieval optimization generation model provided in an embodiment of the present disclosure, combined with Figure 5 As shown in Figure 2, the retrieval optimization generative model is trained through the following steps:

[0106] S502, obtaining at least one training sample, each training sample including a query corpus sample and a label preference annotation result corresponding to the query corpus sample, the label preference annotation result being a comprehensive annotation result obtained by performing preference annotation on the query result output by the large language model and the retrieval result output by the retriever.

[0107] This embodiment constructs a training sample set and uses the training sample set to train the machine learning model, so that the trained machine learning model has good performance. Exemplarily, by constructing sample text and the label preference annotation results corresponding to the sample text as training samples, the retrieval optimization generation model is trained, so that the trained retrieval optimization generation model can optimize the first retrieval result output by the retriever, so that the second retrieval result can effectively align the selection tendency data of the LLM. For the specific method of constructing the training samples in this disclosed embodiment, please refer to the following embodiment and will not be repeated here. Of course, the training samples here can also be preset in advance and can be directly obtained.

[0108] In the embodiment of the present disclosure, there is no limitation on the specific content of the query corpus sample, and the query corpus sample can be Chinese characters, letters or other types of characters. The query corpus sample in the embodiment of the present disclosure includes a comprehensive annotation result obtained by performing preference annotation on the query results output by the large language model corresponding to the query corpus and the retrieval results output by the retriever, which is used to guide the model learning to optimize the synergistic effect of retrieval and generation. Exemplarily, the same group of experts are used to perform preference annotation on a specific plurality of query corpus samples, and after review, etc., the label preference annotation results corresponding to each query corpus sample are finally obtained. Exemplarily, a preference annotation generation strategy is used to generate a label preference annotation for each query corpus sample.

[0109] S504: Generate a predicted query result corresponding to the query corpus sample based on the query corpus sample by using the retrieval optimization generation model.

[0110] The retrieval optimization generation model in this embodiment is a machine learning model, and the present disclosure does not limit the specific architecture of the retrieval optimization generation model. Exemplarily, the retrieval optimization generation model includes at least an encoding module and a prediction module. The encoding module is used to encode the input query corpus sample to obtain encoded features, and the prediction module is used to output the predicted query results corresponding to the query corpus sample based on the encoded features. Optionally, the encoding module includes at least one encoder, and the prediction module includes at least one classification layer or a fully connected layer.

[0111] S506, training the retrieval optimization generation model according to the label preference annotation results and the predicted query results to obtain a trained retrieval optimization generation model, and the trained retrieval optimization generation model is used to optimize the retrieval results output by the retriever.

[0112] In some embodiments, a loss function value is determined based on the difference between the label preference annotation result and the predicted query result, and the retrieval optimization generation model is trained based on the loss function value to obtain a trained retrieval optimization generation model. Exemplarily, the retrieval optimization generation model is trained with the goal of minimizing the loss function value. In some embodiments, when training the retrieval optimization generation model, all modules in the retrieval optimization generation model are trained, or only the parameters of the encoding module or the prediction module are adjusted, while the parameters of other modules remain unchanged.

[0113] The embodiments of the present disclosure do not limit the specific parameter adjustment method, and a forward gradient update method or a reverse gradient update method may be used.

[0114] In some embodiments, the retrieval optimization generation model is trained based on the label preference annotation results and the query results to obtain the trained retrieval optimization generation model, including: determining the first loss function value based on the label preference annotation results and the predicted query results; training the retrieval optimization generation model based on the first loss function value to obtain the trained retrieval optimization generation model.

[0115] For example, the goal of the retrieval optimization generation model is to minimize the loss function L, using the cross entropy loss function:

[0116]

[0117] Among them, m is the number of samples, y i is the true preference label, The label predicted by the model.

[0118] The model in this embodiment approaches the true label using the label probabilities of m samples, minimizing the loss function L, and ultimately learning to reconcile the preferences of the retriever and LLM. This means that through iterative training, the parameters of the retrieval optimization generation model are continuously adjusted to better coordinate the preferences of the retriever and LLM.

[0119] The technical solution provided by the embodiment of the present disclosure obtains at least one training sample including a query corpus sample and a label preference annotation result corresponding to the query corpus sample, and uses the training sample to train the retrieval optimization generation model, so that the trained retrieval optimization generation model can filter and optimize the results returned by the retriever, resolve the preference conflict between the retriever and the LLM, and make the content input to the LLM more accurate by coordinating the differences between the two, thereby ultimately improving the accuracy of the LLM output.

[0120] In some embodiments, Figure 6 A flow chart of a method for obtaining training samples provided in an embodiment of the present disclosure, combined with Figure 6 As shown, the embodiment of the present disclosure provides that obtaining at least one training sample includes the following steps:

[0121] S602: Obtain at least one sample query corpus, where each sample query corpus has a corresponding sample text.

[0122] In this embodiment, there are at least two ways to obtain sample query corpus. First, directly obtain text and obtain the sample query corpus corresponding to the text. Second, directly obtain audio. Use speech-to-text technology to convert speech into text. The first method can fix the content of the text so that the sample text in the final training sample is the expected text content. The second method is to directly extract text based on audio, which simplifies the efficiency of constructing training samples. If the query corpus involves multimodality, for example, it also includes factors such as pictures and videos, the existing speech-to-text and picture-to-text conversion models are used to convert pictures, videos, etc. into sample query corpus that meets the requirements.

[0123] S604 : Determine the tag preference annotation result corresponding to the sample text according to the query result output by the large language model corresponding to the sample query corpus and the retrieval result output by the retriever.

[0124] In some embodiments, based on the query results output by the large language model corresponding to the sample query corpus and the retrieval results output by the retriever, the label preference annotation results corresponding to the sample text are determined, including: determining the preference information of the query results output by the large language model according to the correctness of the query results output by the large language model corresponding to the sample query corpus; determining the preference information of the retrieval results output by the retriever according to the correctness of the query results output by the large language model associated with the retrieval results output by the retriever; determining the comprehensive annotation results of the sample query corpus according to the preference information of the query results output by the large language model and the preference information of the retrieval results output by the retriever; and determining the comprehensive annotation results corresponding to the sample text according to the correspondence between the sample query corpus and the sample text and the comprehensive annotation results of the sample query corpus.

[0125] For example, the LLM’s preference data D is collected by inputting queries to the LLM and marking the LLM’s answers according to their correctness. LLM Through two retrieval results (i.e., the first retrieval result based on the original query and the second retrieval result based on the enhanced query), the retriever’s preference data D is indirectly collected. R It should be noted that the indirect collection of the preference data of the retriever is essentially to mark the preference of the first retrieval result based on the original query input according to the correctness of the output result corresponding to the LLM of the first retrieval result based on the original query, and to mark the preference of the second retrieval result based on the enhanced query input according to the correctness of the output result corresponding to the LLM of the second retrieval result based on the enhanced query. Then, the data is integrated and D LLM and D R Integrate into a unified data set D=D LLM ∪D R .

[0126] In some embodiments, the preference labels can be expanded to a four-value system: strong preference, weak preference, weak no preference, strong no preference to reduce data noise.

[0127] S606: Obtain training samples based on the sample text corresponding to the sample query corpus and the label preference annotation results corresponding to the sample text.

[0128] In this embodiment, the comprehensive data of the large language model answer quality annotation and the retrieval result comparison annotation is used to guide the model to learn to optimize the synergistic effect of retrieval and generation. The ultimate goal is to let the model learn to coordinate the preference differences between the retriever and LLM to make the output of the two more consistent.

[0129] In some embodiments, the current RAG system needs to calculate a large amount of key-value (KV) cache online when processing retrieved documents. This process involves complex matrix operations and data processing, and the demand for computing resources increases significantly. Moreover, online calculation of KV cache not only increases the computational overhead of the system, but also significantly prolongs the time to first token generation (TTFT). For application scenarios with high real-time requirements, such as intelligent customer service, real-time question-and-answer systems, etc., excessively long TTFT will seriously affect the user experience. In addition, a large amount of online calculations also limits the batch size of a single device, resulting in the system being inefficient when processing large-scale queries and unable to meet actual application needs.

[0130] In light of this, the disclosed embodiments employ pre-calculated key-value caching technology. Documents are encoded and KV cached during the offline phase, and stored in vector indexes. During the online phase, the pre-calculated KV cache is directly accessed, reducing online computational overhead and shortening initial token generation time.

[0131] Figure 7 A flowchart of a search enhancement generation method provided by an embodiment of the present disclosure, combined with Figure 7 As shown, the search enhancement generation method of the embodiment of the present disclosure further includes:

[0132] S702, in the offline stage, encode the document in the retriever based on the Transformer encoder to obtain the embedding vector of the document.

[0133] For example, for a document set D = {d1, d2, ..., d N}, use the Transformer-based encoder E (such as Bert) for each document d i Encode and get the embedded vector v i =E(d i ).

[0134] S704: Store the document's embedding vector in a vector retrieval library.

[0135] In this embodiment, the embedding vectors of all documents are stored in the vector index of the retriever's vector retrieval library for rapid subsequent retrieval. Vector retrieval libraries are databases designed specifically for efficient vector storage and search (e.g., FAISS, Milvus), supporting fast similarity calculation. In this embodiment, documents are vectorized and stored in the retrieval library's index. When subsequent search queries are issued, the system converts the user's question into a vector and quickly finds semantically similar document vectors in the library, enabling efficient retrieval.

[0136] S706: Pre-fill the document using the large language model, calculate the key-value matrix of the document and cache it.

[0137] In this embodiment, in addition to storing the embedding vectors of all documents in the vector index of the retriever vector retrieval library in the offline stage for subsequent fast retrieval, the large language model can also store the embedding vectors of each document d i Pre-filling refers to using a large language model (LLM) to process documents in advance, gradually encoding the input text into an intermediate representation (such as a token sequence) that the model can understand, and completing the initial calculation.

[0138] It is understandable that the LLM needs to repeatedly calculate the key and value matrices of the attention mechanism during runtime. The pre-filling stage will pre-calculate the key-value matrices of a large number of documents and cache them. The attention mechanism will be directly called during subsequent reasoning, which greatly reduces the amount of repeated calculations and improves the processing speed of the RAG system. Among them, the key can be a document or document paragraph ID or feature, and the value can be a relevance score, summary, etc. When caching the cache results corresponding to multiple documents, each document has an independent key-value entry in the cache, such as the pre-calculated features of the document (such as keyword matching, vector embedding, etc.), to avoid repeated calculations.

[0139] In some embodiments, Figure 8A A flowchart of a method for outputting a first search result provided by an embodiment of the present disclosure, combined with Figure 8A As shown, the present embodiment provides that inputting the query corpus into the search engine and outputting the first search result may include the following steps:

[0140] S802: Encode the query corpus to obtain a query embedding vector.

[0141] For example, after inputting the query corpus Q, the query corpus Q is encoded to obtain the query embedding vector q emb =E(q).

[0142] S804: Search the vector search library for target embedding vectors whose similarity is greater than a preset threshold according to the query embedding vector to obtain a first search result.

[0143] For example, searching for q in a vector database emb k documents with high similarity D = {d i1 ,d i2 ,…,d ik}. The similarity can be defined by a preset threshold. For example, the query embedding vector q is calculated emb The similarity (e.g., cosine similarity) with all document vectors in the database is calculated, and documents exceeding the threshold are retained. The top-K results are then sorted by similarity in descending order. For example, if the threshold is 0.8, the system will exclude documents with a similarity < 0.8 and return the closest K documents.

[0144] In this embodiment, since the vector retrieval library pre-stores the embedded vectors of all documents, the pre-stored vectors avoid real-time calculation, which greatly improves the retrieval speed of the retriever.

[0145] In some embodiments, Figure 8B A flowchart of a method for outputting query results provided by an embodiment of the present disclosure, combined with Figure 8B As shown, the embodiment provides that the second search result is input into the large language model, and the query result is output, which may include the following steps:

[0146] S806: Determine a key-value matrix of the second search result according to the second search result using the large language model.

[0147] In this embodiment, since the large language model pre-calculates the key-value matrix of a large number of documents and caches them, when obtaining the second search result (such as the retrieved document list), the pre-stored KV cache KV corresponding to these documents is directly extracted. i1 , KV i2 ,…,KV ik , skipping real-time calculations and significantly improving response speed.

[0148] S808 : Input the key-value matrix of the second search result as a pre-filled context into the large language model to generate a pre-filled result.

[0149] In some embodiments, the key-value matrix of the second retrieval result includes key-value matrices corresponding to multiple documents, and the key-value matrix of the second retrieval result is input into the large language model as a pre-filled context to generate a pre-filled result, including: merging the key-value matrices corresponding to the multiple documents to generate an initial context, wherein the initial context includes the original composite location identifier of each document; performing a remapping operation on the location identifier in the key-value cache of each document, wherein the remapping operation includes: replacing the original composite location identifier of each document with a rearranged location identifier; updating the location-related information in the key state based on the remapped location identifier; and inputting the updated key-value cache as a pre-filled context into the large language model to generate a pre-filled result.

[0150] In this embodiment, when the key-value matrix of the second search result includes key-value matrices corresponding to multiple documents, it is necessary to merge the KV caches corresponding to the multiple documents to form a complete context so that the model can understand all the input information at one time. For example, the complete context KV cache KV context =concat(KV i1 ,KV i2 ,…,KV ik ). KV context Enter the LLM to pre-populate and then generate a response.

[0151] In some embodiments, when the KV caches of multiple documents are merged, the token position ID of each document may be disordered after the merge due to changes in the order of the documents. For example, the first token of document A and the first token of document B have the same position ID after the merge, but they actually belong to different documents, which makes the model unable to distinguish the source. Therefore, it is necessary to reallocate a continuous global ID to the cache position of each document. For example, the token position 1-100 of document 1 → new ID 1-100, and the token position 1-50 of document 2 → new ID 101-150. In this way, the global order after splicing (document 1 in front and document 2 in the back) can be directly reflected by the ID, ensuring that the position embedding can correctly reflect the global order after splicing.

[0152] For example, let the original position ID vector be p original , the rearranged position ID vector is p rearranged , achieved through the following formula:

[0153]

[0154] Where l is the length of each document and i is the i-th document.

[0155] Using the above formula, the cache location ID of each document is adjusted from the composite location (original location) [0,…,l,0,…,l,0,…,l] to the rearranged location [0,…,l,l+1,…,2l,2l+1,…,k·l] to ensure the accuracy of the location embedding. The pre-trained LLM is then fine-tuned to adapt it to the new attention mask matrix and location ID settings, significantly improving system performance without compromising model accuracy.

[0156] It should be noted that to ensure the effectiveness of the pre-computed key-value cache, an independent attention mechanism is used to avoid information interference between different documents and ensure accuracy during cache reuse. Specifically, the attention calculation is split into independent sub-modules, each of which processes the key-value data of only one document to avoid global cache invalidation. In specific implementation, the interaction is controlled by the attention mask matrix A: when elements i and j belong to different documents, the corresponding positions in A are set to negative infinity. After softmax, the weights of these positions approach 0, forcing no cross-attention between documents, simulating the isolation effect of the spliced ​​KV cache after retrieval, thereby shielding cross-document attention.

[0157] S810 , generating a new token sequence by decoding based on the pre-filling result.

[0158] S812: Output the query result according to the new token sequence.

[0159] In this embodiment, the pre-calculated KV cache technology transfers complex calculations to the offline stage, and only cache retrieval and splicing are required online, which reduces the amount of online calculations, significantly shortens the time for first token generation, improves computing resource utilization, and enables the RAG system to respond quickly in real-time interactive scenarios, thereby improving user experience.

[0160] In some embodiments, pre-computed KV caching technology can also be applied to retrieval optimization generation models to improve the quality of answers generated by the model while improving efficiency. Figure 9 A flow chart of a method for outputting a second search result provided by an embodiment of the present disclosure, combined with Figure 9 As shown, inputting the first search result into the search optimization generation model and outputting the second search result may include the following steps:

[0161] S902: Input the first search result into the search optimization generation model, and determine the key-value matrix of the first search result according to the first search result.

[0162] In this embodiment, a large number of documents are first pre-processed using pre-calculated KV cache technology to obtain pre-calculated KV cache results to reduce online computing overhead. Subsequently, the pre-calculated KV cache results are input into the RAG pipeline containing the shim model. When the first retrieval result (such as a document fragment) is fed into the retrieval optimization generation model, the retrieval optimization generation model quickly extracts the key-value pairs corresponding to the result through the pre-stored KV cache to form its own key-value matrix. In other words, this step reuses pre-stored data to quickly generate a key-value matrix, reducing the real-time computing burden and improving the response speed.

[0163] S904: Calculate the enhanced weight of the key-value matrix of the first search result using the search optimization generation model.

[0164] In this embodiment, the enhancement weight of each document segment is calculated by analyzing the key-value matrix. The enhancement weight can reflect the importance or relevance of the document segment in the first search result.

[0165] S906: Filter the document paragraphs in the first search result according to the enhanced weight to obtain a document subset.

[0166] In this embodiment, paragraphs are filtered according to weight, and a subset with high relevance is retained. Furthermore, redundant or low-quality content can be eliminated.

[0167] S908: Perform context enhancement on the filtered document subset and output a second search result.

[0168] In this embodiment, the filtered document subset is optimized, for example, by supplementing related information or reorganizing content, to generate a more accurate second search result.

[0169] In this embodiment, the pre-calculated KV cache reduces the amount of online calculations, shortens TTFT, and improves the response speed of the system. When processing the document content corresponding to the pre-calculated KV cache, the retrieval optimization generation model filters and enhances the first retrieval result output by the retriever based on the preference data obtained through its training, further optimizes the information input to the LLM, improves the relevance of the retrieval results and the document screening effect, and thus improves the quality and efficiency of the answer generation of the large language model, effectively improving the performance of the RAG model without increasing too many computing resources.

[0170] In some embodiments, Figure 10 A flowchart of a method for search enhancement generation provided by an embodiment of the present disclosure, combined with Figure 10 As shown, the search enhancement generation method provided in this embodiment includes the following steps:

[0171] S101, obtain query.

[0172] S103: Input the query into the retriever, and the retriever retrieves and outputs a first document paragraph set according to the query.

[0173] In this embodiment, the first document paragraph set is recorded as top-k passages.

[0174] S104: Input the first document paragraph set into the retrieval optimization generation model, and the retrieval optimization generation model assigns a unique identifier to each sentence in the first document paragraph set.

[0175] In this embodiment, the retrieval optimization generation model is pre-trained.

[0176] S105 , based on the query, calculate the relevance score of each sentence to the query.

[0177] S106 , determining whether the relevance score between each sentence and the query is greater than a preset threshold, if so, go to S107 , if not, go to S108 .

[0178] In this embodiment, the preset threshold may be denoted as τ.

[0179] S107 , obtaining a sentence set based on sentences whose relevance scores are greater than a preset threshold.

[0180] In this embodiment, a sentence identification (SID) set S′ having a relevance score higher than a preset threshold is selected.

[0181] S108 , discarding sentences whose relevance scores are less than or equal to a preset threshold.

[0182] S109: Enhancement is performed based on the query and the sentence set to obtain an enhanced query.

[0183] S110: Input the enhanced query into the retriever, and the retriever outputs a second document paragraph set according to the enhanced query.

[0184] In this embodiment, the second document paragraph set is recorded as top-k'passages.

[0185] S111, the retrieval optimization generation model outputs a set of retrieved document paragraphs.

[0186] It should be noted that S111 can be executed after S104 or after S110. That is, the retrieval optimization generation model can output a retrieval document paragraph set based on the first document paragraph set, or it can output a retrieval document paragraph set based on the second document paragraph set. Outputting a retrieval document paragraph set based on the first document paragraph set is the result of a single optimization by the retrieval optimization generation model, while outputting a retrieval document paragraph set based on the second document paragraph set is the result of a secondary optimization by the retrieval optimization generation model.

[0187] S112: Input the retrieved document paragraph set into the large language model.

[0188] S113, the large language model outputs the query result.

[0189] Figure 11 A flowchart of a method for search enhancement generation provided by an embodiment of the present disclosure, combined with Figure 11 As shown, the search enhancement generation method provided in this embodiment includes the following steps:

[0190] S101, obtain query.

[0191] S1031: Encode the query to obtain a query vector.

[0192] S1032: Utilize a search engine to search in a vector database according to the query vector to obtain a first document paragraph set.

[0193] S1121, obtaining a key-value cache based on the retrieved document paragraph set.

[0194] In this embodiment, the process of determining the retrieved document paragraph set may refer to the above S104 to S111, which will not be described in detail in this embodiment.

[0195] S1122: The key-value cache is spliced ​​to form a complete context.

[0196] S1123, pre-filling the large language model based on the complete context.

[0197] S113, the large language model outputs the query result.

[0198] Figure 12 A flowchart of a method for search enhancement generation provided by an embodiment of the present disclosure, combined with Figure 12 As shown, the search enhancement generation method provided in this embodiment also includes the following steps:

[0199] S121, obtaining a document collection.

[0200] S122a, in the offline stage, the document collection is offline encoded to obtain the embedding vector of the document.

[0201] S123a, storing the document's embedding vector in a vector retrieval library.

[0202] In this embodiment, the vector retrieval database serves as an external knowledge base, and the retriever can directly retrieve document fragments related to the query from the vector retrieval database, thereby improving the retrieval efficiency of the retriever.

[0203] S122b, in the offline phase, pre-fill each document in the document collection using the large language model.

[0204] S123b, calculate the key value cache of each document and save it.

[0205] In this embodiment, the saved key-value cache can be reused in subsequent reasoning to reduce repeated calculations. It should be noted that S122a-S123a and S122b-S123b can be executed in parallel or sequentially, and this embodiment does not limit the order of sequential execution.

[0206] Figure 13 A flowchart of a method for search enhancement generation provided by an embodiment of the present disclosure, combined with Figure 13 As shown, the search enhancement generation method provided in this embodiment includes the following steps:

[0207] S101, obtain query.

[0208] S1031: Encode the query to obtain a query vector.

[0209] S1032: Utilize a search engine to search in a vector database according to the query vector to obtain a first document paragraph set.

[0210] S1041: Acquire a key-value cache based on the first document paragraph set.

[0211] S1042: Pre-fill the retrieval optimization generation model based on the key-value cache corresponding to the first document paragraph set.

[0212] S1043: Outputting the retrieved document paragraphs based on the first document paragraph set using the retrieval optimization generation model.

[0213] In this embodiment, the process of determining the retrieved document paragraph set may refer to S105 to S111 above, and will not be described in detail in this embodiment.

[0214] S112: Input the retrieved document paragraph set into the large language model.

[0215] S113, the large language model outputs the query result.

[0216] In this embodiment, the integration of a search optimization generation model and pre-computed KV caching technology optimizes data flow and component collaboration within the system, enhancing the system's robustness to noise and erroneous data and improving stability. Furthermore, the system can better support real-time applications and large-scale data processing, while also enabling personalized services to meet the needs of a wider range of scenarios.

[0217] Based on the same inventive concept, the present disclosure also provides a search enhancement generation device, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0218] Figure 14 A schematic diagram of a search enhancement generation device according to an embodiment of the present disclosure is shown. Figure 14 As shown, the device includes: an acquisition module 141 for acquiring a query corpus; a first retrieval module 142 for inputting the query corpus into a retriever and outputting a first retrieval result; a second retrieval module 143 for inputting the first retrieval result into a retrieval optimization generation model and outputting a second retrieval result; wherein the retrieval optimization generation model is trained based on historical query corpus and query results corresponding to the historical query corpus, and the query results corresponding to the historical query corpus are output by a large language model; a query module 144 for inputting the second retrieval result into the large language model and outputting a query result.

[0219] In some embodiments, the second retrieval module 143 is specifically used to: assign a unique identifier to each retrieval statement in the first retrieval result to obtain a first retrieval statement set; calculate the similarity between each retrieval statement in the first retrieval statement set and the query corpus to obtain a relevance score between each retrieval statement and the query corpus; select retrieval statements with a relevance score greater than a preset threshold to obtain a second retrieval statement set; and concatenate the second retrieval statement set with the query corpus to obtain a second retrieval result.

[0220] In some embodiments, the second retrieval module 143 is further used to: input the second retrieval result into the retriever and output a third retrieval result; input the third retrieval result into the retrieval optimization generation model, assign a unique identifier to each retrieval statement in the third retrieval result, and obtain a third retrieval statement set; calculate the similarity between each retrieval statement in the third retrieval statement set and the query corpus, and obtain a relevance score between each retrieval statement and the query corpus; select retrieval statements with a relevance score greater than a preset threshold to obtain a fourth retrieval statement set; splice the fourth retrieval statement set with the query corpus to obtain a fourth retrieval result; inputting the second retrieval result into the large language model and outputting the query result includes: inputting the fourth retrieval result into the large language model and outputting the query result.

[0221] In some embodiments, the second retrieval module 143 is further used to: obtain at least one training sample, each training sample including a query corpus sample and a label preference annotation result corresponding to the query corpus sample, the label preference annotation result being a comprehensive annotation result obtained by performing preference annotation on the query result output by the large language model and the retrieval result output by the retriever; generate a predicted query result corresponding to the query corpus sample based on the query corpus sample through the retrieval optimization generation model; train the retrieval optimization generation model based on the label preference annotation result and the predicted query result to obtain a trained retrieval optimization generation model, and the trained retrieval optimization generation model is used to optimize the retrieval results output by the retriever.

[0222] In some embodiments, the second retrieval module 143 is further used to: obtain at least one sample query corpus, each sample query corpus having a corresponding sample text; determine the label preference annotation results corresponding to the sample text based on the query results output by the large language model corresponding to the sample query corpus and the retrieval results output by the retriever; obtain the training sample based on the sample text corresponding to the sample query corpus and the label preference annotation results corresponding to the sample text.

[0223] In some embodiments, the second retrieval module 143 is further used to: determine preference information of the query result output by the large language model according to the correctness of the query result output by the large language model corresponding to the sample query corpus; determine preference information of the retrieval result output by the retriever according to the correctness of the query result output by the large language model associated with the retrieval result output by the retriever; determine the comprehensive annotation result of the sample query corpus according to the preference information of the query result output by the large language model and the preference information of the retrieval result output by the retriever; determine the comprehensive annotation result corresponding to the sample query corpus according to the correspondence between the sample query corpus and the sample text and the comprehensive annotation result of the sample query corpus.

[0224] In some embodiments, the second retrieval module 143 is further used to: determine a first loss function value based on the label preference annotation result and the predicted query result; and train the retrieval optimization generation model based on the first loss function value to obtain the trained retrieval optimization generation model.

[0225] In some embodiments, a key-value cache module is also included, which is used to: in the offline stage, encode the document in the retriever based on the Transformer encoder to obtain the embedding vector of the document; store the embedding vector of the document in a vector retrieval library; use a large language model to pre-fill the document, calculate the key-value matrix of the document and cache it.

[0226] In some embodiments, the first retrieval module 142 is specifically configured to: encode the query corpus to obtain a query embedding vector; and retrieve a target embedding vector having a similarity greater than a preset threshold in the vector retrieval library based on the query embedding vector to obtain a first retrieval result.

[0227] In some embodiments, the query module 144 is specifically used to: determine the key-value matrix of the second retrieval result based on the second retrieval result using a large language model; input the key-value matrix of the second retrieval result as a pre-filled context into the large language model to generate a pre-filled result; generate a new token sequence based on the pre-filled result by decoding; and output the query result according to the new token sequence.

[0228] In some embodiments, the key-value matrix of the second retrieval result includes key-value matrices corresponding to multiple documents, and the query module 144 is specifically used to: merge the key-value matrices corresponding to the multiple documents to generate an initial context, wherein the initial context includes the original composite location identifier of each document; remap the original composite location identifier of each document to replace the original composite location identifier of each document with a rearranged location identifier; update the location-related information in the key state based on the remapped location identifier; and input the updated key-value cache as a pre-filled context into the large language model to generate a pre-filled result.

[0229] In some embodiments, the second retrieval module is specifically used to: input the first retrieval result into a retrieval optimization generation model, and determine the key-value matrix of the first retrieval result based on the first retrieval result; use the retrieval optimization generation model to calculate the enhancement weight of the key-value matrix of the first retrieval result; filter the document paragraphs in the first retrieval result according to the enhancement weight to obtain a document subset; perform context enhancement on the document subset, and output a second retrieval result.

[0230] It should be noted that the examples and application scenarios implemented by the modules in the above-mentioned apparatus embodiment are the same as those implemented by the corresponding steps in the method embodiment, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules, as part of the apparatus, can be executed in a computer system, such as a set of computer-executable instructions.

[0231] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0232] Based on the same inventive concept, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any of the above-described search enhancement generation methods by executing the executable instructions. Since the principles for solving the problem in this electronic device embodiment are similar to those in the above-described method embodiment, the implementation of this electronic device embodiment can refer to the implementation of the above-described method embodiment, and repeated parts will not be repeated here.

[0233] Refer to the following Figure 15 15 to describe the electronic device 1500 according to this embodiment of the present disclosure. Figure 15 The electronic device 1500 shown is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0234] like Figure 15 As shown, electronic device 1500 is implemented as a general-purpose computing device. Components of electronic device 1500 may include, but are not limited to, the aforementioned at least one processing unit 1510, the aforementioned at least one storage unit 1520, and a bus 1530 connecting various system components (including storage unit 1520 and processing unit 1510).

[0235] The storage unit stores program code, which can be executed by the processing unit 1510, so that the processing unit 1510 performs the steps described in the "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 1510 can perform the following steps of the above-mentioned method embodiment: obtaining a query corpus; inputting the query corpus into a search engine and outputting a first search result; inputting the first search result into a pre-trained search optimization generation model and outputting a second search result; wherein the search optimization generation model is trained based on historical query corpus and query results corresponding to the historical query corpus, and the query results corresponding to the historical query corpus are obtained by outputting a large language model; inputting the second search result into the large language model and outputting a query result.

[0236] The storage unit 1520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 15201 and / or a cache memory unit 15202 , and may further include a read-only memory unit (ROM) 15203 .

[0237] The storage unit 1520 may also include a program / utility 15204 having a set (at least one) of program modules 15205, such program modules 15205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0238] Bus 1530 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0239] Electronic device 1500 can also communicate with one or more external devices 1540 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1500, and / or any device that enables electronic device 1500 to communicate with one or more other computing devices (e.g., a router, modem, etc.). Such communication can occur via input / output (I / O) interface 1550. Furthermore, electronic device 1500 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 1560. As shown, network adapter 1560 communicates with other modules of electronic device 1500 via bus 1530. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with electronic device 1500, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0240] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0241] Based on the same inventive concept, embodiments of the present disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the aforementioned search enhancement generation methods. Because the principles underlying the problems solved by this computer-readable storage medium embodiment are similar to those of the aforementioned method embodiment, the implementation of this computer-readable storage medium embodiment can be referenced to the implementation of the aforementioned method embodiment, and any repetitions will not be repeated.

[0242] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0243] In the present disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0244] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0245] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a standalone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0246] Based on the same inventive concept, embodiments of the present disclosure further provide a computer program product, including a computer program or instructions, which, when executed by a processor, implements the search enhancement generation method of any one of the above-described method embodiments. Because the principles for solving the problems of this computer program product embodiment are similar to those of the above-described method embodiments, the implementation of this computer program product embodiment can be referenced to the implementation of the above-described method embodiments, and any repetitions will not be repeated here.

[0247] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0248] Furthermore, although the steps of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0249] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0250] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

Claims

1. A search enhancement generation method, characterized in that: The method comprises: Get query corpus; Input the query corpus into the search engine and output a first search result; Inputting the first search result into a search optimization generation model and outputting a second search result; wherein the search optimization generation model is trained based on historical query corpus and query results corresponding to the historical query corpus, and the query results corresponding to the historical query corpus are output by a large language model; The second search result is input into the large language model, and a query result is output.

2. The search enhancement generation method according to claim 1, characterized in that: The step of inputting the first search result into a search optimization generation model and outputting a second search result includes: assigning a unique identifier to each search statement in the first search result to obtain a first search statement set; Calculating the similarity between each search statement in the first search statement set and the query corpus to obtain a relevance score between each search statement and the query corpus; Selecting search statements with relevance scores greater than a preset threshold to obtain a second search statement set; The second search statement set is concatenated with the query corpus to obtain a second search result.

3. The search enhancement generation method according to claim 1 or 2, characterized in that: The step of concatenating the second search statement set with the query corpus to obtain a second search result further includes: inputting the second search result into a retriever and outputting a third search result; Inputting the third search result into the search optimization generation model, assigning a unique identifier to each search statement in the third search result, and obtaining a third search statement set; Calculating the similarity between each search statement in the third search statement set and the query corpus to obtain a relevance score between each search statement and the query corpus; Selecting search statements with relevance scores greater than a preset threshold to obtain a fourth search statement set; concatenating the fourth search statement set with the query corpus to obtain a fourth search result; The step of inputting the second search result into the large language model and outputting a query result includes: The fourth search result is input into the large language model, and a query result is output.

4. The search enhancement generation method according to claim 1, characterized in that: The retrieval optimization generation model is trained in the following manner: Obtain at least one training sample, each training sample including a query corpus sample and a label preference annotation result corresponding to the query corpus sample, where the label preference annotation result is a comprehensive annotation result obtained by performing preference annotation on the query result output by the large language model and the retrieval result output by the retriever; Generate predicted query results corresponding to the query corpus samples through the retrieval optimization generation model based on the query corpus samples; According to the label preference annotation results and the predicted query results, the retrieval optimization generation model is trained to obtain a trained retrieval optimization generation model. The trained retrieval optimization generation model is used to optimize the retrieval results output by the retriever.

5. The search enhancement generation method according to claim 4, characterized in that: The obtaining of at least one training sample comprises: Obtain at least one sample query corpus, each of the sample query corpus having a corresponding sample text; Determining a label preference annotation result corresponding to the sample text based on the query result output by the large language model corresponding to the sample query corpus and the retrieval result output by the retriever; The training sample is obtained according to the sample text corresponding to the sample query corpus and the label preference annotation result corresponding to the sample text.

6. The search enhancement generation method according to claim 5, characterized in that: Determining the label preference annotation result corresponding to the sample text according to the query result output by the large language model corresponding to the sample query corpus and the retrieval result output by the retriever includes: Determining preference information of the query result output by the large language model based on the correctness of the query result output by the large language model corresponding to the sample query corpus; determining preference information of the retrieval results output by the retriever based on the correctness of the query results output by the large language model and associated with the retrieval results output by the retriever; Determining a comprehensive annotation result of the sample query corpus based on the preference information of the query result output by the large language model and the preference information of the retrieval result output by the retriever; The comprehensive annotation result corresponding to the sample text is determined according to the correspondence between the sample query corpus and the sample text and the comprehensive annotation result of the sample query corpus.

7. The search enhancement generation method according to claim 4, characterized in that: The method of training the retrieval optimization generation model according to the label preference annotation results and the query results to obtain the trained retrieval optimization generation model includes: Determining a first loss function value according to the label preference annotation result and the prediction query result; The retrieval optimization generation model is trained according to the first loss function value to obtain the trained retrieval optimization generation model.

8. The search enhancement generation method according to claim 1, characterized in that: Also includes: In the offline stage, the documents in the retriever are encoded based on the Transformer encoder to obtain the embedding vector of the document; Storing the embedding vector of the document in a vector retrieval library; The documents are pre-populated using a large language model, and the key-value matrix of the documents is calculated and cached.

9. The search enhancement generation method according to claim 8, characterized in that: The step of inputting the query corpus into a search engine and outputting a first search result includes: Encoding the query corpus to obtain a query embedding vector; According to the query embedding vector, a target embedding vector having a similarity greater than a preset threshold is searched in the vector retrieval library to obtain a first retrieval result.

10. The search enhancement generation method according to claim 8, characterized in that: The step of inputting the second search result into the large language model and outputting a query result includes: Determining a key-value matrix of the second search result based on the second search result using a large language model; Inputting the key-value matrix of the second search result as a pre-filled context into the large language model to generate a pre-filled result; Based on the pre-filled result, a new token sequence is generated by decoding; According to the new token sequence, the query result is output.

11. The search enhancement generation method according to claim 10, characterized in that: The key-value matrix of the second search result includes key-value matrices corresponding to multiple documents, and inputting the key-value matrix of the second search result as a pre-filled context into the large language model to generate a pre-filled result includes: Merging the key-value matrices corresponding to the multiple documents to generate an initial context, wherein the initial context includes an original composite location identifier of each document; Performing a remapping operation on the original composite position identifier of each document, replacing the original composite position identifier of each document with the rearranged position identifier; updating the position-related information in the key state based on the remapped position identifier; The updated key-value cache is used as the pre-population context and input into the large language model to generate pre-population results.

12. The search enhancement generation method according to any one of claims 8 to 10, characterized in that: The step of inputting the first search result into a search optimization generation model and outputting a second search result includes: Inputting the first search result into a search optimization generation model, and determining a key-value matrix of the first search result according to the first search result; Calculating the enhanced weight of the key-value matrix of the first search result using the search optimization generation model; Filtering the document paragraphs in the first search result according to the enhanced weight to obtain a document subset; Context enhancement is performed on the document subset, and a second search result is output.

13. A search enhancement generation device, characterized in that: The device comprises: Acquisition module, used to obtain query corpus; A first retrieval module, configured to input the query corpus into a search engine and output a first retrieval result; a second retrieval module, configured to input the first retrieval result into a retrieval optimization generation model and output a second retrieval result; wherein the retrieval optimization generation model is trained based on historical query corpus and query results corresponding to the historical query corpus, and the query results corresponding to the historical query corpus are output by a large language model; The query module is used to input the second search result into the large language model and output a query result.

14. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the search enhancement generation method according to any one of claims 1 to 12 by executing the executable instructions.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the search enhancement generation method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, it implements the search enhancement generation method according to any one of claims 1 to 12.

Images