A text retrieval enhancement generation method, device, medium and equipment
By constructing a golden section vector database and introducing adversarial training and noise classification mechanisms, the problem of insufficient noise robustness in existing technologies is solved, and the generation accuracy and robustness of the model in noisy environments are improved.
Patent Information
- Application Number
- CN202510546788.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-28
- Publication Date
- 2025-12-26
- Estimated Expiration
- 2045-04-28
AI Technical Summary
Existing retrieval augmentation techniques fail to adequately reflect the complex retrieval environment of the real world in terms of robustness to noise, leading to inaccurate or incorrect answers generated by the models.
By constructing a golden ratio vector database containing key information annotations and introducing adversarial training and noise classification mechanisms, the robustness of the model in noisy environments is improved. Specific steps include screening and vectorizing known corpora to generate an initial vector database; extracting adversarial examples from external databases; adding a noise classification layer and an attention fusion layer to the large-scale language model; and optimizing model parameters through adaptive adversarial training.
This improves the model's robustness in noisy environments and the accuracy of generated results, ensuring that the model can identify and distinguish different types of noise and generate higher-quality answers.
Smart Images
Figure CN120104718B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of retrieval augmented generation, and particularly relates to a text retrieval augmented generation method, device, medium and equipment. BACKGROUND
[0002] Retrieval-Augmented Generation (RAG) is an effective solution to alleviate the limitations of Large Language Models (LLMs) in specific domains or tasks by integrating external database knowledge. Although RAG performs well in enhancing the generation ability of models, inappropriate retrieval passages may potentially hinder the ability of LLMs to generate comprehensive and high-quality answers. For example, irrelevant information or low-quality content retrieved may introduce noise, leading to inaccurate or even incorrect answers generated by the model. However, RAG research on retrieval noise robustness fails to fully reflect the complex retrieval environment in the real world, thereby limiting its effectiveness in practical applications. SUMMARY
[0003] Therefore, in order to solve the technical problems in the prior art, the present application provides a text retrieval augmented generation method, device, medium and equipment.
[0004] The present application provides a text retrieval augmented generation method, comprising:
[0005] Performing vectorization operation on known corpus to generate initial vector database; performing screening, labeling and vectorization operation on known corpus to generate golden split vector database containing key information labeling;
[0006] Obtaining training questions contained in a data set used for training language large model, extracting adversarial samples including relevant retrieval noise, irrelevant retrieval noise and counterfactual retrieval noise from external database according to relevance with the training questions; recalling first corpus related to the training questions from the golden split vector database; fusing the first corpus and the adversarial samples to obtain fusion training set;
[0007] Adding a classification layer for classifying noise types before the output layer of the language large model and an attention fusion layer connected to the output end of the classification layer to obtain an improved language large model; training the improved language large model using the fusion training set;
[0008] The user question is obtained, a second corpus related to the user question is recalled from an initial vector database, the user question and the second corpus are input into a trained language large model, and a noise category of the second corpus is output through a classification layer; the noise category output by the classification layer is taken as a generation condition, and the user question and the second corpus are input into an attention fusion layer to perform feature fusion through an attention mechanism to generate a fused context representation; and a weight distribution or a decoding strategy is dynamically adjusted according to the noise category by an output layer, and the fused context is decoded through the dynamically adjusted weight distribution or decoding strategy to generate an answer.
[0009] Further, the known corpus is filtered, specifically including:
[0010] The known corpus is cut into corpus blocks, and key feature vectors including keywords, topics, entities and sentence structures are extracted from the corpus blocks;
[0011] At least one evaluation index score of corpus block information density, source authority and relevance to the search topic is calculated based on the key feature vectors to obtain a value score of each corpus block; wherein the evaluation index score of the corpus block information density is obtained by measuring the distribution and importance of the keywords in the corpus block through TF-IDF and BM25 algorithm, the evaluation index score of the source authority is obtained based on the number of references of the source and the weight of the publication channel, and the evaluation index score of the relevance of the search topic is obtained by calculating the degree of relevance to the search topic using cosine similarity;
[0012] The corpus block with a value score higher than a set threshold is taken as a high-value corpus block.
[0013] Further, the relevant samples including the relevant search noise, the irrelevant search noise and the counterfactual search noise are extracted from the external database according to the relevance to the training question, specifically including:
[0014] For each training question, a plurality of candidate contexts are retrieved from an external knowledge base using a pre-trained search model;
[0015] The context with the highest semantic similarity to the current training question but with a missing answer field is selected as a relevant search noise sample from the candidate contexts;
[0016] A candidate context irrelevant to the current training question is randomly selected from the search results of other training questions as an irrelevant search noise sample;
[0017] A candidate context containing a correct answer is randomly selected from the search results of the current training question, and an error information is replaced with an answer entity to generate a counterfactual search noise sample.
[0018] Further, the recalling the second corpus related to the user question from the initial vector database specifically comprises:
[0019] Performing lexical complexity analysis and sentence structure analysis on the user question by using natural language processing technology to obtain the complexity label of the user question: {simple, medium, complex};
[0020] Selecting a retrieval strategy matching the complexity of the user question: the complexity label of the user question selects the retrieval strategy: for simple questions, keyword matching retrieval strategy is adopted; for medium questions, similarity calculation retrieval strategy based on vector space model is used; for complex questions, semantic retrieval strategy based on deep learning model BERT is used;
[0021] According to the retrieval strategy, the second corpus related to the user question is recalled from the initial vector database.
[0022] Further, a classification layer for classifying noise types is added before the output layer of the language large model, and an attention fusion layer connected to the output end of the classification layer;
[0023] The classification layer is a multi-layer feedforward neural network structure, including an input layer receiving the last transformer layer hidden state of the language model, a plurality of fully connected layers constituting an intermediate layer, and an output layer outputting the probability distribution of noise types through a softmax activation function; the attention fusion layer uses a multi-head attention mechanism, a weighted fusion mechanism and a residual connection to perform attention fusion on the noise.
[0024] Further, the improved language large model is trained using the fusion training set, taking the sum of noise classification error and answer relevance error as the loss function, and optimizing the parameters of the Prompt language large model through gradient reversal.
[0025] Further, the output layer dynamically adjusts the weight distribution or decoding strategy according to the noise category, and decodes the fused context through the dynamically adjusted weight distribution or decoding strategy to generate an answer, specifically comprising:
[0026] Calling a conditional weight set matching the noise category from a pre-trained weight database, the conditional weight set containing decoding parameter configurations for different noise features;
[0027] During the decoding process, the similarity scores of the current time step context representation and all possible text blocks are calculated; the similarity scores are multiplied by the corresponding conditional weights to adjust the selection probability of the text blocks associated with the context, and an initial answer is generated;
[0028] Performing semantic correction, logical consistency verification and noise filtering on the initial answer to generate a final answer.
[0029] The application provides a text retrieval enhancement generation device, comprising:
[0030] A corpus construction module is configured to perform vectorization on a known corpus to generate an initial vector database, and perform screening, labeling and vectorization on the known corpus to generate a golden split vector database containing key information labels.
[0031] A data enhancement module is configured to obtain training questions contained in a data set used for training a language large model, extract adversarial samples including relevant retrieval noise, irrelevant retrieval noise and counterfactual retrieval noise from an external database according to the relevance to the training questions, recall first corpus related to the training questions from the golden split vector database, and fuse the first corpus and the adversarial samples to obtain a fusion training set.
[0032] A training module is configured to add a classification layer for classifying noise types before an output layer of the language large model and an attention fusion layer connected to an output end of the classification layer to obtain an improved language large model, and train the improved language large model using the fusion training set.
[0033] An answer generation module is configured to obtain a user question, recall second corpus related to the user question from the initial vector database, input the user question and the second corpus into the trained language large model, output noise categories of the second corpus through the classification layer, input the noise categories output by the classification layer as a generation condition into the attention fusion layer together with the user question and the second corpus, perform feature fusion through an attention mechanism to generate a fused context representation, and output a layer to dynamically adjust a weight distribution or a decoding strategy according to the noise categories, and decode the fused context through the dynamically adjusted weight distribution or decoding strategy to generate an answer.
[0034] The application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned text retrieval enhancement generation method.
[0035] The application provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned text retrieval enhancement generation method when executing the program.
[0036] The above-mentioned at least one technical scheme adopted by the application can achieve the following beneficial effects:
[0037] In the text retrieval enhancement generation method provided by the application, first, a golden split vector database containing key information annotations is constructed by screening, annotating and vectorizing known corpus, to ensure that the first corpus recalled by the training problem has higher quality, to guide the model to pay more attention to important features in the training process, and improve the understanding and answering ability of the model; then the golden split vector database is fused with the relevant retrieval noise, irrelevant retrieval noise and counterfactual retrieval noise to form a fusion training set, and a large language model with a noise classification task is trained through the fusion training set, so that the model can identify and distinguish different types of noise, enhance the adaptability of the model to different noise, and help the model learn how to distinguish relevant and irrelevant information, so as to be more robust when processing actual problems. BRIEF DESCRIPTION OF DRAWINGS
[0038] The drawings described herein are used to provide further understanding of the application, and form a part of the application. The illustrative embodiments of the application and their descriptions serve to explain the application, and do not constitute an improper limitation on the application. In the drawings:
[0039] Figure 1 A text retrieval enhancement generation method flowchart is provided for the application;
[0040] Figure 2 A text retrieval enhancement generation step frame diagram is provided for the application;
[0041] Figure 3 A text retrieval enhancement generation device diagram is provided for the application. DETAILED DESCRIPTION
[0042] To make the purpose, technical scheme and advantages of the application clearer, the technical scheme of the application will be described clearly and completely below in combination with specific embodiments of the application and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the specification, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.
[0043] To make the above-mentioned purpose, features and advantages of the application more obvious and easy to understand, the application will be further described in detail below in combination with the drawings and specific embodiments.
[0044] The RAG research on retrieval noise robustness is often limited to a limited set of noise types, which fails to fully reflect the complex retrieval environment in the real world, thereby limiting its effectiveness in practical applications. Retrieval noise in the real world can include relevant but redundant information, completely irrelevant content, and even counter-factual information that contradicts facts. These different types of noise pose higher requirements for the robustness of LLMs. Based on this, the present invention aims to address the shortcomings of the prior art by providing a retrieval enhancement generation method based on adaptive adversarial training and noise classification. By introducing adversarial training and noise classification mechanisms, the robustness of the model in a noisy environment is improved, and the accuracy and reliability of the generated results are optimized.
[0045] Figure 1 and Figure 2 The text retrieval enhancement generation method flow and step framework shown in the figure illustrates the technical solutions provided by the embodiments of the present application. Specifically, the following steps are included:
[0046] S1: Perform vectorization operation on known corpus to generate initial vector database; perform filtering, labeling and vectorization operation on known corpus to generate golden split vector database containing key information labeling. Specifically, the following steps are included:
[0047] S11: Use a period as a cutting mark to perform cutting operation on the known corpus.
[0048] S12: Select an existing embedding model capable of embedding text to perform embedding operation on the corpus block, obtain the vector representation of each corpus block and store it in any database such as PostgreSQL.
[0049] S13: Construct a golden split database by filtering high-quality corpus blocks and labeling their relevance to form a high-quality initial database.
[0050] In a specific embodiment, the step of filtering high-quality corpus blocks in step S13 includes:
[0051] S131: Based on the initial vector database, extract key features from the corpus block, including but not limited to keywords, topics, entities, sentence structure, and convert the key features into feature vectors.
[0052] S132: Calculate the value score for each corpus block according to the predefined scoring criteria, which include at least one evaluation indicator of relevance of the corpus block to the search topic, information density, source authority. Among them, the evaluation indicator score of information density of the corpus block is obtained by measuring the distribution and importance of keywords in the corpus block through TF-IDF and BM25 algorithm, the evaluation indicator score of source authority is obtained based on the number of references of the source and the weight of the publication channel, and the evaluation indicator score of the relevance of the search topic is obtained by using cosine similarity or semantic matching degree method to calculate the relevance degree with the search topic.
[0053] S133: Set a score threshold, and screen the corpus blocks with value scores higher than the score threshold as high-value corpus blocks.
[0054] S2: Obtain a training question included in a data set for training a language large model, extract an adversarial sample including relevant search noise, irrelevant search noise and counterfactual search noise from an external database according to the relevance to the training question; recall a first corpus related to the training question from a golden split vector database; fuse the first corpus and the adversarial sample to obtain a fused training set. Specifically, it includes:
[0055] S21: Initialize the retrieval model and the language model, and use the pre-trained retrieval model DPR and the language model LLaMA-2 as the basic architecture.
[0056] S22: Retrieve candidate contexts from an external database, and for each query, use the retrieval model to retrieve multiple candidate contexts from an external knowledge base.
[0057] S23: Generate relevant search noise samples, select the most relevant context to the query (but not containing the correct answer) from the retrieved candidate contexts as the relevant search noise sample; generate irrelevant search noise samples, randomly select a context irrelevant to the current query from the search results of other queries as the irrelevant search noise sample; generate counterfactual search noise samples, randomly select one from the retrieved context containing the correct answer and replace the answer entity with false information to generate the counterfactual search noise sample.
[0058] S24: Construct an adversarial sample set, combine the generated relevant search noise samples, irrelevant search noise samples and counterfactual search noise samples with the golden search context (containing the correct answer) to form the adversarial sample set.
[0059] Among them, the step of recalling the first corpus related to the training question from the golden split vector database includes:
[0060] Indexing: In the golden section vector database, each corpus is converted into a vector representation and indexed for fast retrieval.
[0061] Query vector representation: Convert the training question into a vector representation, which is usually done through a language model, such as using BERT, GPT, etc. model to get the embedding vector of the question.
[0062] Similarity calculation: Calculate the similarity between the training question vector and all corpus vectors in the database. This can be done through cosine similarity, Euclidean distance or other distance measurement methods.
[0063] Ranking and selection: Sort the corpus according to the similarity score and select the top N most relevant corpus as the "recall" result.
[0064] S3: Add a classification layer for classifying noise types before the output layer of the language large model and an attention fusion layer connected to the output end of the classification layer to obtain an improved language large model; train the improved language large model using the fusion training set. Specifically, it includes:
[0065] S31: Design auxiliary tasks through a multi-task learning mechanism to enable the model to identify and distinguish different types of noise. The large language model is a classification layer for classifying noise types added before the output layer of the Prompt language large model and an attention fusion layer connected to the output end of the classification layer.
[0066] Among them, the classification layer is a multi-layer feedforward neural network structure, including an input layer that receives the last transformer layer hidden state of the language model, a middle layer composed of multiple (2-3) fully connected layers, each followed by BatchNorm and ReLU activation functions, and an output layer that outputs the probability distribution of noise types through the softmax activation function. Specific parameter design: The first layer maps the model hidden state (assuming dimension d_model) to a smaller dimension (such as d_model / 2), the middle layer maintains the same dimension or gradually decreases, and the output layer has a dimension equal to the predefined number of noise types N. Regularization techniques: Dropout (0.1-0.3) is applied after each fully connected layer, and weight regularization (L2 regularization) is used to prevent overfitting.
[0067] The attention fusion layer uses a multi-head attention mechanism, a weighted fusion mechanism, and a residual connection to fuse the noise. The multi-head attention mechanism uses the classification layer output as the query vector, uses the language model final layer hidden state as the key and value vectors, and the number of heads can be set to 4-8, and the dimension of each head is d_model / number of heads. The weighted fusion mechanism assigns different attention weights to different types of noise based on the classification results, and designs a gating mechanism to dynamically adjust the attention distribution according to the noise classification probability. The residual connection design adds a residual connection between the original language model output and the attention processed output, which includes a learnable parameter α to control the weight ratio of the original output and the processed output. The output layer normalization applies LayerNorm after the attention output to ensure a stable output distribution.
[0068] S32: Adopting an adaptive adversarial training strategy, the generated adversarial samples are injected into the training process.
[0069] S33: Optimize the model parameters through gradient reversal, so that they remain robust in a noisy environment.
[0070] In a specific embodiment, step S31 includes:
[0071] S311: Construct a retrieval enhancement model, and use the retrieval enhancement framework DPR to retrieve context information related to the question from an external knowledge base.
[0072] S312: Classify the retrieved context, and according to the relevance to the question, divide the retrieved context into three types: context related to the question but not containing the correct answer (relevant retrieval noise), context irrelevant to the question (irrelevant retrieval noise), and context related to the theme of the question but containing false information (counterfactual retrieval noise).
[0073] S313: Train the classification model, and use the annotated question and retrieval context dataset to train a classification model to identify different types of retrieval noise.
[0074] S314: Input a new question, input the new question into the retrieval enhancement model, retrieve the relevant context and classify it.
[0075] S315: According to the output of the classification model, determine the category of the retrieval context corresponding to the question, and provide classification information for the subsequent generation task.
[0076] In a specific embodiment, step S32 includes:
[0077] S321: Based on the golden section database, use the adversarial sample generation module to dynamically generate relevant retrieval noise, irrelevant retrieval noise, and counterfactual retrieval noise.
[0078] S322: Mix the generated adversarial samples with the original training data, adopt an adaptive adversarial training strategy, and optimize the model through gradient reversal or adversarial loss function.
[0079] S323: Design a multi-task learning mechanism and introduce a noise classification task as an auxiliary task to enable the model to identify and distinguish relevant retrieval noise, irrelevant retrieval noise and counterfactual retrieval noise.
[0080] S324: Jointly optimize the loss functions of the main task and the auxiliary task, i.e., use the sum of noise classification classification error and answer relevance error as the loss function to adjust the model parameters, and improve the robustness and generation accuracy of the model in the noise environment.
[0081] S4: Obtain the user question, recall the second corpus related to the user question from the initial vector database; input the user question and the second corpus into the trained language large model, output the noise category of the second corpus through the classification layer; the noise category output by the classification layer is used as a generation condition, and the user question and the second corpus are input into the attention fusion layer, and the feature fusion is performed through the attention mechanism to generate the fused context representation; the output layer dynamically adjusts the weight distribution or decoding strategy according to the noise category, and decodes the fused context through the dynamically adjusted weight distribution or decoding strategy to generate the answer. Specifically, it includes:
[0082] S41: Classify the user input question, and select a retrieval strategy according to the question type and complexity.
[0083] S42: Execute the retrieval strategy and obtain the relevant corpus.
[0084] S43: Input the retrieved corpus and the user question into the large language model, and design prompt words to guide the model to generate high-quality answers.
[0085] In a specific embodiment, step S41 includes:
[0086] S411: Question complexity analysis, analyze the input question using natural language processing technology, including vocabulary complexity and sentence structure, to evaluate the complexity of the question. Mark the question as simple, medium or complex through a pre-trained classification model, and identify key words.
[0087] S412: Retrieval strategy selection, select retrieval strategy according to problem complexity. For simple problems, use keyword matching. For medium problems, use vector space model for similarity calculation. For complex problems, call deep learning model BERT for semantic retrieval. BERT is a pre-trained natural language processing model that can understand the deep meaning and contextual relationship in language. The steps of semantic retrieval using BERT include: preprocessing the question raised by the user, inputting the preprocessed input question into the BERT model, outputting the contextual embedding vector of each word in the input question, calculating the similarity between the contextual embedding vector and the pre-stored document vector in the database to realize recall.
[0088] S413: Dynamic adjustment, relevance feedback is performed on the retrieval results, and if the keyword matching effect is not ideal, the vector space model retrieval is switched.
[0089] S414: According to the selected strategy, retrieve relevant corpus from the initial vector database and the golden section database. Through the retrieval of the two databases, high-quality corpus matching the problem complexity is ensured to support subsequent text generation.
[0090] In a specific embodiment, step S43 comprises:
[0091] S431: For each user question, recall the second corpus related to the user question from the initial vector database.
[0092] S42: Input the user question and the second corpus into the trained large language model, and identify the noise category to which the second corpus belongs through the noise classification layer in the model.
[0093] S433: The identified noise category is input as a condition and fused with the user question and the second corpus through an attention mechanism to pass to the output layer of the large language model.
[0094] S434: Dynamically adjust the weights or parameters of the output layer according to the noise category to guide the generation of answers that adapt to the noise category. In the process of generating answers, the model dynamically adjusts the weights or parameters of the output layer according to the noise category. Specifically, different noise categories correspond to different conditional weights, and a set of conditional weights matching the noise category is called from a pre-established weight database, the set of conditional weights includes decoding parameter configurations for different noise features, and these conditional weights are applied in the decoding process to dynamically adjust the decoding process: the output layer applies the conditional weights to adjust the selection probability of the answer text block at each time step of decoding to generate answers that adapt to the noise category.
[0095] S435: After generating the answer, post-process the generated text based on the noise category, including spelling correction, semantic correction, and logical consistency check, to improve the accuracy and context consistency of the answer.
[0096] S436: Based on the prompt word guide model, generate a high-quality answer based on the retrieved corpus. The prompt word can be any prompt word that can make the large language model answer the user's question based on the retrieved corpus. Here is an example:
[0097] "I will give you a user input question and some text blocks related to the question as reference information below. Please answer the user's question based on the text blocks provided, and strictly rely on the reference information provided to answer the user's question.
[0098] Let's start formally:
[0099] User question: <question>
[0100] Reference information:
[0101] Text chunk: <Text Chunk>.
[0102] The text retrieval enhancement generation method provided by one or more embodiments of the present application is based on the same idea. The present application also provides a corresponding text retrieval enhancement generation device as shown in the accompanying drawings, which comprises: Figure 3
[0103] A corpus construction module is configured to perform vectorization operation on the known corpus to generate an initial vector database, and perform screening, labeling and vectorization operation on the known corpus to generate a golden split vector database containing key information labels.
[0104] A data enhancement module is configured to obtain training questions contained in a data set for training a language large model, extract adversarial samples including relevant retrieval noise, irrelevant retrieval noise and counterfactual retrieval noise from an external database according to the relevance to the training questions, recall first corpus related to the training questions from the golden split vector database, and fuse the first corpus and the adversarial samples to obtain a fusion training set.
[0105] A training module is configured to add a classification layer for classifying noise types before the output layer of the language large model and an attention fusion layer connected to the output end of the classification layer to obtain an improved language large model, and train the improved language large model using the fusion training set.
[0106] An answer generation module is configured to obtain a user question, recall second corpus related to the user question from the initial vector database, input the user question and the second corpus into the trained language large model, output the noise category of the second corpus through the classification layer, input the noise category output by the classification layer as a generation condition into the attention fusion layer together with the user question and the second corpus, perform feature fusion through the attention mechanism to generate a fused context representation, and output a layer that dynamically adjusts the weight distribution or decoding strategy according to the noise category, and decodes the fused context through the dynamically adjusted weight distribution or decoding strategy to generate an answer.
[0107] The specific limitations of the text retrieval enhancement generation device can be referred to the limitations of the text retrieval enhancement generation method described above, which will not be repeated here. Each module in the above text retrieval enhancement generation device can be realized by software, hardware and their combination in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0108] The application further provides a computer readable storage medium, which stores a computer program. Figure 1 The application provides a text retrieval enhancement generation method.
[0109] The application further provides a computer device structure, which comprises a processor, an internal bus, a network interface, a memory and a nonvolatile memory at a hardware level, and can further comprise other hardware required by a business. The processor reads a corresponding computer program from the nonvolatile memory into the memory and then runs to implement the above-mentioned Figure 1 The application provides a text retrieval enhancement generation method.
[0110] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a nonvolatile computer readable storage medium and can include the processes of the above-mentioned embodiments when executed. Any reference to a memory, storage, database or other medium in the embodiments of the application can include at least one of a nonvolatile and volatile memory. The nonvolatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory or an optical memory. The volatile memory can include a random access memory (RAM) or an external cache memory. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM).
[0111] The technical features of the above-mentioned embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as the combinations of the technical features do not contradict, they should be considered as the range disclosed by the application.< / question>
Claims
1. A method for text retrieval enhancement generation, the method comprising: The method comprises the following steps: vectorizing the known corpus to generate an initial vector database; screening, labeling and vectorizing the known corpus to generate a golden split vector database containing key information labels; obtaining training questions contained in a data set for training a language large model, extracting adversarial samples including relevant retrieval noise, irrelevant retrieval noise and counterfactual retrieval noise from an external database according to the relevance to the training questions; recalling first corpus related to the training questions from the golden split vector database; fusing the first corpus and the adversarial samples to obtain a fusion training set; adding a classification layer for classifying noise types before the output layer of the language large model and an attention fusion layer connected to the output end of the classification layer to obtain an improved language large model; training the improved language large model using the fusion training set; obtaining a user question and recalling second corpus related to the user question from the initial vector database; inputting the user question and the second corpus into the trained language large model, and outputting the noise category of the second corpus through the classification layer; inputting the noise category output by the classification layer as a generation condition into the attention fusion layer together with the user question and the second corpus, performing feature fusion through the attention mechanism to generate a fused context representation; and outputting a layer that dynamically adjusts the weight distribution or decoding strategy according to the noise category, and decoding the fused context through the dynamically adjusted weight distribution or decoding strategy to generate an answer; The screening of the known corpus specifically comprises: cutting the known corpus into corpus blocks, and extracting key feature vectors including keywords, topics, entities and sentence structures from the corpus blocks; calculating at least one evaluation index score of corpus block information density, source authority and relevance to the retrieval topic based on the key feature vectors to obtain a value score of each corpus block; wherein the evaluation index score of the corpus block information density is obtained by measuring the distribution and importance of keywords in the corpus block through TF-IDF and BM25 algorithm, the evaluation index score of the source authority is obtained based on the number of citations of the source and the weight of the publication channel, and the evaluation index score of the relevance to the retrieval topic is obtained by calculating the degree of relevance to the retrieval topic using cosine similarity; corpus blocks with a value score higher than a set threshold are regarded as high-value corpus blocks; The extraction of adversarial samples including relevant retrieval noise, irrelevant retrieval noise and counterfactual retrieval noise from the external database according to the relevance to the training questions specifically comprises: for each training question, a pre-trained retrieval model is used to retrieve a plurality of candidate contexts from an external knowledge base; selecting a context with the highest semantic similarity to the current training question but with a missing answer field as a relevant retrieval noise sample from the candidate contexts; randomly selecting a candidate context unrelated to the current training question from the retrieval results of other training questions as an irrelevant retrieval noise sample; randomly selecting a candidate context containing the correct answer from the retrieval results of the current training question, replacing the answer entity with false information to generate a counterfactual retrieval noise sample; and The output layer dynamically adjusts the weight distribution or decoding strategy according to the noise category, and decodes the fused context through the dynamically adjusted weight distribution or decoding strategy to generate an answer, specifically comprising: Call the conditional weight set matched with the noise category from the pre-trained weight database, which contains decoding parameter configurations for different noise characteristics; Calculate the similarity scores of the context representation of the current time step and all possible text blocks during the decoding process; multiply the similarity scores with the corresponding conditional weights to adjust the selection probability of the text blocks associated with the context, and generate an initial answer; Perform semantic correction, logical consistency verification and noise filtering on the initial answer to generate a final answer.
2. The method of claim 1, wherein, The second corpus related to the user question is recalled from the initial vector database, specifically comprising: Perform vocabulary complexity analysis and sentence structure analysis on the user question to obtain the complexity label of the user question: {simple, medium, complex}; Select a retrieval strategy matched with the complexity of the user question: the complexity label of the user question selects the retrieval strategy: keyword matching retrieval strategy is used for simple questions; similarity calculation retrieval strategy based on vector space model is used for medium questions; semantic retrieval strategy based on deep learning model BERT is used for complex questions; Recall the second corpus related to the user question from the initial vector database according to the retrieval strategy.
3. The method of claim 1, wherein, A classification layer for classifying noise types is added before the output layer of the language large model, and an attention fusion layer connected with the output end of the classification layer is added; The classification layer is a multi-layer feedforward neural network structure, including an input layer receiving the hidden state of the last transformer layer of the language model, a middle layer composed of multiple fully connected layers, and an output layer outputting the probability distribution of noise types through a softmax activation function; the attention fusion layer uses a multi-head attention mechanism, a weighted fusion mechanism and a residual connection to perform attention fusion on the noise.
4. The method of claim 3, wherein, The improved language large model is trained using the fusion training set, taking the sum of noise classification error and answer relevance error as the loss function, and optimizing the parameters of the Prompt language large model through gradient reversal.
5. A text retrieval enhancement generation apparatus characterized by comprising: Comprise: A corpus construction module for vectorizing known corpora to generate an initial vector database; Filter, label and vectorize the known corpora to generate a golden split vector database containing key information labels; A data enhancement module for obtaining training questions in the data set used to train the language large model, extracting adversarial samples including relevant retrieval noise, irrelevant retrieval noise and counterfactual retrieval noise from an external database according to the relevance to the training questions; recalling first corpus related to the training question from the golden split vector database; fuse the first corpus with the adversarial samples to obtain a fusion training set; A training module for adding a classification layer for classifying noise types before the output layer of the language large model, and an attention fusion layer connected with the output end of the classification layer to obtain an improved language large model; Train the improved language large model using the fusion training set; The answer generation module is configured to obtain a user question, recall a second corpus related to the user question from an initial vector database, input the user question and the second corpus into the trained language large model, and output a noise category of the second corpus through a classification layer; The noise category output by the classification layer is taken as a generation condition, and is input into an attention fusion layer together with the user question and the second corpus, feature fusion is performed through an attention mechanism, a fused context representation is generated, a weight distribution or a decoding strategy is dynamically adjusted by the output layer according to the noise category, and the fused context is decoded through the dynamically adjusted weight distribution or decoding strategy to generate an answer; The known corpus is filtered, and specifically includes: The known corpus is cut into corpus blocks, and key feature vectors including keywords, topics, entities and sentence structures are extracted from the corpus blocks; At least one evaluation index score of information density, source authority and relevance to a search topic of the corpus block is calculated based on the key feature vectors to obtain a value score of each corpus block; wherein the evaluation index score of the information density of the corpus block is obtained by measuring the distribution and importance of the keywords in the corpus block through TF-IDF and BM25 algorithms, the evaluation index score of the source authority is obtained based on the number of references of the source and the weight of the publication channel, and the evaluation index score of the relevance of the search topic is obtained by calculating the relevance degree to the search topic using cosine similarity. The corpus block with a value score higher than a set threshold is taken as a high-value corpus block. The relevant search noise, irrelevant search noise and counterfactual search noise are extracted from the external database according to the relevance to the training question, and specifically include: For each training question, a plurality of candidate contexts are retrieved from an external knowledge base using a pre-trained search model; A context with the highest semantic similarity to the current training question but with a missing answer field is selected from the candidate contexts as a relevant search noise sample; A candidate context unrelated to the current training question is randomly selected from the search results of other training questions as an irrelevant search noise sample; A candidate context containing a correct answer is randomly selected from the search results of the current training question, and an error information is replaced with an answer entity to generate a counterfactual search noise sample; The output layer dynamically adjusts the weight distribution or the decoding strategy according to the noise category, and decodes the fused context through the dynamically adjusted weight distribution or decoding strategy to generate an answer, and specifically includes: A condition weight set matched with the noise category is called from a pre-trained weight database, and the condition weight set contains decoding parameter configurations for different noise features; In the decoding process, the similarity scores of the current time step context representation and all possible text blocks are calculated; the similarity scores are multiplied by the corresponding condition weights to adjust the selection probability of the text blocks associated with the context, and an initial answer is generated; The initial answer is subjected to semantic correction, logical consistency verification and noise filtering to generate a final answer.
6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-4.
7. A computer device, comprising: Computer program product comprising a memory, a processor and a computer program stored on the memory and loadable into the processor, the processor implementing the method according to any one of claims 1 to 4 when running the program.