Retrieval enhancement generation method

Abstract

The invention discloses a retrieval enhancement generation method, which relates to the technical field of natural language processing, and comprises the following steps of: preliminary retrieval: firstly, performing preliminary retrieval from a preset external knowledge base by using keyword matching, a TF-IDF algorithm or a BM25 algorithm based on query input by a user; the main purpose of the preliminary retrieval stage is to quickly screen out a document set related to user query. Fine filtering: on the basis of a document set obtained by preliminary retrieval, performing semantic analysis on document contents by adopting a natural language processing technology, and dynamic weight adjustment: in a retrieval process, dynamically adjusting a retrieval weight by a system according to a historical query record and a click behavior of a user. For example, for themes or keywords that the user often pays attention to, the system will endow higher weights, thereby optimizing the retrieval results. And sorting the retrieval results: finally, sorting the retrieval results by the system according to the relevance scores of the documents, and presenting the documents which most accord with the query intention of the user to a generation module for use.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for retrieval enhancement generation. Background Art

[0002] In the field of natural language processing (NLP), generative models such as the GPT series have demonstrated strong performance in many tasks, such as text generation, dialogue systems, and machine translation. These generative models usually rely on a large amount of pre-training data. Although the amount of data and domain range involved in the model training are huge, they still face certain challenges when dealing with specific fields or tasks that require precise information. These challenges are mainly manifested in the lack of accuracy of generated content, poor relevance of content, and the possibility that the model generates answers that do not meet user expectations when dealing with complex and diverse needs.

[0003] In order to make up for the shortcomings of pure generative models, researchers proposed the Retrieval-augmented Generation (RAG) method. RAG combines the advantages of information retrieval and text generation, retrieves relevant information from external knowledge bases, and then generates more accurate and coherent content based on this information. This method improves the performance of generative models in specific tasks to a certain extent, especially when a large amount of external knowledge needs to be introduced, such as question-answering systems and knowledge-driven dialogue systems.

[0004] Existing retrieval-enhanced generation methods also have certain limitations.

[0005] The accuracy of the retrieval module directly affects the quality of the generated content. Current retrieval technology is mainly based on keyword matching. Although the retrieval efficiency can be improved through the term frequency-inverse document frequency (TF-IDF) or BM25 algorithm, it often fails to accurately capture the user's true intention when processing semantically complex queries.

[0006] The generation module has limited ability to integrate the retrieved document content, which can easily lead to insufficient coherence of the generated content or information conflicts.

[0007] Existing methods perform poorly in dynamically adapting to changes in user needs and are unable to adjust retrieval and generation strategies in a timely manner based on user feedback.

[0008] There is an urgent need for an improved retrieval enhancement generation method to improve the accuracy of information retrieval, the coherence of generated content, and the ability to dynamically adapt to changes in user needs. The present invention aims to address the deficiencies in these prior arts by introducing a multi-level retrieval algorithm, a complex frequency domain multi-layer perceptron model, and a dynamic feedback mechanism to achieve a more effective retrieval enhancement generation method and improve the overall performance of natural language processing tasks. Summary of the invention

[0009] The purpose of the present invention is to provide a method for retrieval enhancement generation in view of the deficiencies in the prior art.

[0010] A method for retrieval enhancement generation, comprising the following parts:

[0011] A1: Information retrieval module, which retrieves relevant documents from a preset external knowledge base or the Internet based on the query entered by the user;

[0012] A2: Information processing module, which pre-processes the retrieved documents, including denoising, deduplication, and text summary generation;

[0013] A4: Generation module, which uses a complex frequency domain multi-layer perceptron to perform in-depth analysis on the processed documents and generate text content that meets user needs;

[0014] A5: Feedback mechanism, dynamically adjust the retrieval and generation algorithms through user feedback on generated content.

[0015] In some embodiments, the complex frequency domain multilayer perceptron includes the following parts:

[0016] B1: Input layer, which receives the processed text data or feature vector and passes it as input to the subsequent hidden layers;

[0017] B2: Hidden layer, the core part of the model, composed of multiple perceptron layers cascaded together, each layer includes frequency domain processing units;

[0018] B3: Output layer, the generated text content is generated by the output layer.

[0019] In some embodiments, each frequency domain processing unit in the hidden layer first performs Fourier transform on the input data to convert the time domain signal into a frequency domain signal. By extracting and analyzing frequency domain features, the model can identify potential patterns and periodic features in the text.

[0020] In some embodiments, the complex frequency domain multilayer perceptron further includes in step B3:

[0021] B31: The output layer integrates the processing results of the hidden layer and converts the frequency domain information back to the time domain through inverse Fourier transform to generate text that conforms to natural language grammar and context logic.

[0022] In some embodiments, the information retrieval module includes the following steps:

[0023] A11: Preliminary search: Based on the query entered by the user, a preliminary search is performed from the preset external knowledge base using keyword matching, TF-IDF algorithm or BM25 algorithm;

[0024] A12: Fine filtering: Based on the document set obtained from the initial retrieval, natural language processing technology is used to perform semantic analysis on the document content, and pre-trained language models such as BERT are used for contextual semantic matching to screen out more relevant and accurate documents.

[0025] The advantages of the present invention compared with the prior art are:

[0026] Compared with the traditional MLP model, the innovation of the CF-MLP generation model is that it introduces frequency domain processing technology, which enables the model to better capture the complex semantic structure and periodic patterns in the text. In addition, through the dual processing mechanism of frequency domain and time domain, the model shows higher accuracy and coherence when processing complex text generation tasks, especially when it is necessary to generate highly consistent long text content. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is the overall architecture diagram of this solution;

[0028] Figure 2 This is the flowchart of the multi-level retrieval algorithm in this scheme;

[0029] Figure 3 This is a structural diagram of the complex frequency domain multilayer perceptron generation model in this scheme. DETAILED DESCRIPTION

[0030] Combined with Figure 1-3 As shown,

[0031] Embodiment 1

[0032] The information retrieval module in the present invention is the core part of the retrieval-augmented generation method. Its workflow is as follows:

[0033] Initial search: First, based on the query entered by the user, a preliminary search is performed from the preset external knowledge base using keyword matching, TF-IDF (term frequency-inverse document frequency) algorithm or BM25 algorithm. The main purpose of the initial search stage is to quickly filter out the document collection related to the user's query.

[0034] Fine filtering: Based on the document collection obtained through preliminary retrieval, natural language processing (NLP) technology is used to perform semantic analysis on the document content, and pre-trained language models such as BERT are used for contextual semantic matching to filter out more relevant and accurate documents. This process aims to reduce the interference of irrelevant documents and improve the quality of retrieval results.

[0035] Dynamic weight adjustment: During the search process, the system will dynamically adjust the search weight based on the user's historical query records and click behavior. For example, the system will assign a higher weight to topics or keywords that users often pay attention to, thereby optimizing the search results.

[0036] Sorting retrieval results: Finally, the system sorts the retrieval results according to the relevance scores of the documents and presents the documents that best meet the user's query intent to the generation module for use.

[0037] Complex frequency domain multilayer perceptron generation model:

[0038] The Complex-Frequency Domain Multilayer Perceptron (CF-MLP) generative model is one of the core innovations of this invention, which is specially designed to improve the performance of generative models in dealing with complex text generation tasks. This model combines the structural advantages of frequency domain analysis technology and multilayer perceptron (MLP), and enhances the model's ability to understand and express complex semantics and contextual information by performing frequency domain processing on input data. The following is a detailed description of the model:

[0039] Model structure,CF-MLP generation model consists of the following key parts:

[0040] Input Layer: Receives the processed text data or feature vector and passes it as input to the subsequent hidden layers.

[0041] Hidden Layers: The core part of the model, composed of multiple cascaded perceptron layers. Each layer includes a frequency domain processing unit, which is responsible for performing frequency domain processing operations such as Fourier transform on the input data to extract frequency features in the data. This processing method can better capture periodic patterns and complex semantic relationships in the text.

[0042] Frequency Domain Processing Unit: Each frequency domain processing unit in the hidden layer first performs Fourier transform on the input data to convert the time domain signal into a frequency domain signal. Then, by extracting and analyzing the frequency domain features, the model can identify the potential patterns and periodic features in the text, thereby improving the coherence and accuracy of the generated results.

[0043] Output Layer: The final generated text content is generated by the output layer. The output layer integrates the processing results of the hidden layer and converts the frequency domain information back to the time domain through inverse Fourier transform, thereby outputting text content that conforms to the context logic.

[0044] The algorithm implementation and CF-MLP generation model are mainly carried out through the following steps:

[0045] Frequency domain conversion: In each perceptron unit of the hidden layer, the input feature vector is first Fourier transformed. This step converts the input data from the time domain to the frequency domain so that frequency features can be extracted.

[0046] Feature extraction and processing: After frequency domain conversion, the model analyzes the high-frequency and low-frequency information in the frequency domain signal to identify periodic patterns in the text. This process enhances the model's ability to understand complex semantic structures, especially when processing long texts or content with complex dependencies.

[0047] Reverse transformation: In the output layer, the model transforms the generation results in the frequency domain back to the time domain through inverse Fourier transform to generate text that conforms to natural language grammar and contextual logic.

[0048] Innovative structure: Compared with the traditional MLP model, the innovation of the CF-MLP generation model is that it introduces frequency domain processing technology, which enables the model to better capture the complex semantic structure and periodic patterns in the text. In addition, through the dual processing mechanism of frequency domain and time domain, the model shows higher accuracy and coherence when processing complex text generation tasks, especially when it is necessary to generate highly consistent long text content.

[0049] Protection points,CF-MLP generation model protection points mainly include:

[0050] Design and implementation of the frequency domain processing unit: protect the specific structure and processing flow of the unit in the hidden layer, especially the application of Fourier transform in text feature extraction.

[0051] Combination of Multilayer Perceptron and Frequency Domain Processing: Protect the overall structural design that combines frequency domain processing technology with the traditional MLP model and its application in generative models.

[0052] The reverse frequency domain conversion process of generated text: The protection model uses the inverse Fourier transform in the output layer to convert the frequency domain information into time domain text.

[0053] Through these innovative designs, the CF-MLP generative model shows significant superiority in the field of natural language processing, especially in complex generation tasks that require high accuracy and coherence, and has broad application prospects.

[0054] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0055] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0057] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for retrieval enhancement generation, characterized in that: Includes the following sections: A1: Information retrieval module, which retrieves relevant documents from a preset external knowledge base or the Internet based on the query entered by the user; A2: Information processing module, which pre-processes the retrieved documents, including denoising, deduplication, and text summary generation; A4: Generation module, which uses a multi-layer perceptron in the complex frequency domain to perform in-depth analysis on the processed documents and generate text content that meets user needs; A5: Feedback mechanism, dynamically adjust the retrieval and generation algorithms through user feedback on generated content.

2. The method for retrieval enhancement generation according to claim 1, characterized in that: The complex frequency domain multilayer perceptron includes the following parts: B1: Input layer, which receives the processed text data or feature vector and passes it as input to the subsequent hidden layers; B2: Hidden layer, the core part of the model, composed of multiple perceptron layers cascaded together, each layer includes frequency domain processing units; B3: Output layer, the generated text content is generated by the output layer.

3. The method for retrieval enhancement generation according to claim 2, characterized in that: Each frequency domain processing unit in the hidden layer first performs Fourier transform on the input data to convert the time domain signal into a frequency domain signal. By extracting and analyzing the frequency domain features, the model can identify the potential patterns and periodic features in the text.

4. The method for retrieval enhancement generation according to claim 3, characterized in that: The complex frequency domain multilayer perceptron also includes in step B3: B31: The output layer integrates the processing results of the hidden layer and converts the frequency domain information back to the time domain through inverse Fourier transform to generate text that conforms to natural language grammar and context logic.

5. The method for retrieval enhancement generation according to claim 4, characterized in that: The information retrieval module comprises the following steps: A11: Preliminary search: Based on the query entered by the user, a preliminary search is performed from the preset external knowledge base using keyword matching, TF-IDF algorithm or BM25 algorithm; A12: Fine filtering: Based on the document set obtained from the initial retrieval, natural language processing technology is used to perform semantic analysis on the document content, and pre-trained language models such as BERT are used for contextual semantic matching to screen out more relevant and accurate documents.

Images