Generation method for improving quality of content generated by RAG technology based on semantic matching

Abstract

The invention discloses a generation method for improving the quality of content generated by an RAG technology based on semantic matching, and the method comprises the steps: dividing text data provided by a user into sub-texts, converting the sub-texts and a question into vectors through an embedding machine, storing the vectors into a vector database, and retrieving a plurality of knowledge fragments most related to the question; performing word segmentation on the question and each knowledge fragment and mapping the question and each knowledge fragment into vectors; positioning word segmentation pairs with semantic matching; calculating cosine similarity of the matching vector pairs, and quantifying local correlation by introducing a weighting loss function of a temperature parameter adjusting key term weight; and finally, aggregating the loss values of all segmented words to obtain an overall correlation score of the knowledge fragment. And sorting knowledge fragments according to scores, rigidly removing low-correlation fragments with loss values exceeding a threshold value, and constructing residual high-correlation fragments and questions into a final prompt input language model to generate a response. According to the method, through word segmentation-level semantic matching, it is ensured that knowledge fragments input into the language model are strongly related to question cores.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to a method for improving the quality of content generated by RAG technology based on semantic matching. Background Technology

[0002] In recent years, Large Language Models (LLMs) have made groundbreaking progress in the field of Natural Language Processing, with their text generation, reasoning, and knowledge understanding capabilities sparking a technological revolution. Retrieval Augmented Generative (RAG) technology, as a core extension of LLM, significantly improves the model's generation accuracy and factual consistency in professional fields (such as medicine and law) by constructing a searchable knowledge base from user-provided text data. However, existing RAG technologies suffer from the following key drawbacks: 1. Invalid information contamination: Top-k knowledge fragments retrieved by RAG from external data often contain content weakly related to the question's semantics (such as redundant background descriptions and locally conflicting information). 2. Interference mechanisms: Case 1: A user asks "The effect of aspirin on kidney function," and RAG returns a fragment containing "the pharmacological mechanism of aspirin," ignoring the core term "kidney function." Case 2: The question "The color of Napoleon's horse" is interfered with by the long text "Napoleon's life," diluting the key information "black." These additional invalid information can cause LLM to generate incorrect answers due to noise. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for improving the quality of content generated by RAG technology based on semantic matching.

[0004] The objective of this invention is achieved through the following technical solution: a method for improving the quality of content generated by RAG technology based on semantic matching, the method comprising:

[0005] S1. The text data provided by the user is segmented into subtexts and input together with the question into the embedding machine to be converted into embedding vectors. The embedding vectors of the subtexts and the embedding vectors of the question are stored together in the vector database and knowledge fragments are filtered out.

[0006] S2. Segment the questions and knowledge fragments into words, obtain question word segments and knowledge fragment word segments, and map them into vectors. Find the vector that matches the question vector in each knowledge fragment. Calculate the similarity between the matching word segment vectors, as well as the similarity between all knowledge fragment word segment vectors and all question word segment vectors. Calculate the overall loss of the knowledge fragments as the relevance result based on the similarity.

[0007] S3. Extract the sentence set based on the relevance results and the preset relevance threshold, and send it along with the question to the prompt template to form the final input prompt. Input the input prompt into the large language model to obtain the response result.

[0008] Furthermore, the process of selecting knowledge fragments specifically includes: finding the j sub-text vectors that are closest to the question vector in the vector database, and using the knowledge fragments corresponding to these sub-text vectors as valid knowledge fragments provided by RAG.

[0009] Furthermore, the question and knowledge fragment are segmented into words to obtain question words and knowledge fragment words, which are then mapped into vectors. A word segmenter is used to segment the words and map the question words and knowledge fragment words into vectors.

[0010] Furthermore, in calculating the similarity between the matched word segmentation vectors, cosine similarity is used to calculate the similarity between the two vectors.

[0011] Furthermore, in the similarity between all knowledge fragment word vectors and all question word vectors, cosine similarity is used to calculate the similarity of each word vector.

[0012] Furthermore, the overall loss of the knowledge fragment calculated through similarity as a relevance result includes:

[0013] First, calculate the weighted loss:

[0014] ;

[0015] in, This represents the similarity between the matched word segmentation vectors. This represents the similarity between the word segmentation vectors of knowledge fragments and the word segmentation vectors of questions. For temperature parameters; For knowledge fragments The Middle Each word segmentation is for Similarity weights;

[0016] The weighted loss of all word segmentation pairs is averaged to obtain the overall loss of the knowledge segment.

[0017] Furthermore, the step of extracting a set of sentences based on relevance combination and a preset relevance threshold includes:

[0018] The sentences are sorted from high to low scores based on the relevance results. Sentences with an overall loss higher than h are discarded based on a set relevance threshold h. The remaining set of sentences and the question are sent to the prompt template to form the final input prompt.

[0019] On the other hand, this invention also provides a system for implementing the method, the system comprising: a preprocessing module, the word segmenter, a relevance module, and a postprocessing module;

[0020] The preprocessing module is used to segment the user-provided text data into subtexts and input them together with the question into the embedding machine to convert them into embedding vectors. The embedding vectors of the subtexts and the embedding vectors of the question are stored together in the vector database and knowledge fragments are filtered out.

[0021] The word segmenter is used to segment the question and knowledge fragment into words, resulting in question word segments and knowledge fragment word segments;

[0022] The relevance module is used to map question word segmentation and knowledge fragment word segmentation into vectors, find the vector that matches the question vector in each knowledge fragment, calculate the similarity between the matched word segmentation vectors, and the similarity between all knowledge fragment word segmentation vectors and all question word segmentation vectors. The overall loss of the knowledge fragment is calculated as the relevance result based on the similarity.

[0023] The post-processing module is used to extract a set of sentences based on the relevance results and a preset relevance threshold, and send them together with the question to the prompt template to form the final input prompt. The input prompt is then input into the large language model to obtain the response result.

[0024] On the other hand, this invention also provides a generation apparatus for improving the quality of RAG technology-generated content based on semantic matching, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it implements the generation method for improving the quality of RAG technology-generated content based on semantic matching.

[0025] On the other hand, the present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the aforementioned method for improving the quality of RAG-generated content based on semantic matching.

[0026] The beneficial effects of this invention are:

[0027] Precise knowledge supply: Through word-segmentation-level semantic matching, ensure that the knowledge fragments input into the language model are strongly related to the core of the problem.

[0028] Reduce hallucination responses: Filter low-relevance segments, such as discarding noisy content with loss values ​​higher than the threshold h, cut off the transmission path of erroneous knowledge, and reduce the probability of the model providing fictitious answers.

[0029] Adjusting temperature parameters to improve sensitivity and increase the weight of core term matching. Attached Figure Description

[0030] Figure 1 This invention provides a method for improving the quality of RAG (Rapid Algorithm Generation) technology-generated content based on semantic matching.

[0031] Figure 2This is a flowchart of word segmentation-level relevance calculation provided in an embodiment of the present invention;

[0032] Figure 3 A post-processing flowchart provided for embodiments of the present invention;

[0033] Figure 4 This is a schematic diagram of a generation device for improving the quality of RAG technology-generated content based on semantic matching, provided as an embodiment of the present invention. Detailed Implementation

[0034] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0035] Referring to Figures 1-3, the method for improving the quality of RAG technology content generation based on semantic matching includes the following steps:

[0036] S1. As shown in Figure 1, RAG retrieval first retrieves the additional text data provided by the user. Split into subtext After and the problem Input together to the embedded machine Embedded machine Convert text to vector form The original text was then stored together with the vector database, and subsequently, the vectors matching the question vectors were retrieved from the vector database. Similar data vectors, and using a pre-designed template. The corresponding subtext data and the question are input into the language model. The specific steps are as follows:

[0037] S1.1, User provides additional text data and problems and text data and problems Input into the language model;

[0038] S1.2, The model first processes the text data. Input to splitter In, and using a divider Will Divided into Subtext ,Right now in and the segmented Subtext and problems Input to embedded machine middle;

[0039] S1.3, Embedded Machine k subtexts Convert to k embedding vectors Store in a vector database Among them and the problem It is also converted into an embedding vector. Stored together with the original question text in a vector database. middle;

[0040] S1.4 Find the vector in the vector database that matches the problem vector. The j nearest subtext vectors in the vector space The knowledge fragments corresponding to these sub-text vectors are used as valid knowledge fragments provided by RAG.

[0041] S2, such as Figure 2 As shown, before sending the final prompt to the model, we need to send the knowledge fragments and questions given by the RAG retrieval system to the word segmenter. The word segmenter will decompose the knowledge fragments and questions into word segments (for example, "Napoleon's horse is black" will be decomposed into "Napoleon's horse", "is", and "black"). Then, the relevance results between each knowledge fragment and question are calculated.

[0042] S2.1, the problem The input is fed into the word segmenter to obtain the segmented word for the problem. ;

[0043] S2.2, sequentially collect all the knowledge fragments. The input is fed into the word segmenter to obtain segmented knowledge fragments. ;

[0044] S2.3 First, a word segmenter will be used to segment the question into words. and Mapped to vector and ;

[0045] S2.4 Find the vector In each knowledge fragment Matching vectors and To locate isomorphic semantic units across texts; for example, the matching vectors for the question "What color was Napoleon's horse?" and the knowledge fragment "Napoleon's horse was black" should be: "Napoleon's horse" matches "Napoleon's horse", "is" matches "is", and "what color" matches "black".

[0046] S2.5 Calculate the similarity between matching vectors, using cosine similarity to calculate the similarity between two vectors:

[0047]

[0048] Computational knowledge All word segmentation and question word segmentation vectors Similarity:

[0049]

[0050] S2.6 Calculate the weighted loss, where for The Middle Each word segmentation is for Similarity weights, Temperature parameter:

[0051]

[0052] The knowledge fragment is obtained by averaging the local losses of all word segmentation pairs. Overall loss:

[0053]

[0054] S2.7, The relevance results of all sentences were obtained. Then, it is sent to the third step for post-processing.

[0055] S3, such as Figure 3 As shown, based on the correlation results calculated in step 2, we perform a final processing step on the RAG results to resolve the issue. Together they form the final hint. This prompt After the input is given to the model, the model generates the final response.

[0056] S3.1. Sort all sentences according to relevance results, and sort sentences from high to low scores based on relevance scores to ensure that highly relevant content is input into the model first:

[0057]

[0058] S3.2, Based on the threshold Overall losses Higher than The sentences that are discarded are the set of sentences that remain. This set and the problem Send to prompt template The final input prompt is formed as follows:

[0059]

[0060] S3.3, will prompt Input to the model Then, the model generates a response. This is the final response result of the model.

[0061]

[0062] Based on the above-disclosed method embodiments, a generation system for improving the quality of RAG technology-generated content based on semantic matching is also provided, including: a preprocessing module, the word segmenter, a relevance module, and a postprocessing module;

[0063] The preprocessing module is used to segment the user-provided text data into subtexts and input them together with the question into the embedding machine to convert them into embedding vectors. The embedding vectors of the subtexts and the embedding vectors of the question are stored together in the vector database and knowledge fragments are filtered out.

[0064] The word segmenter is used to segment the question and knowledge fragment into words, resulting in question word segments and knowledge fragment word segments;

[0065] The relevance module is used to map question word segmentation and knowledge fragment word segmentation into vectors, find the vector that matches the question vector in each knowledge fragment, calculate the similarity between the matched word segmentation vectors, and the similarity between all knowledge fragment word segmentation vectors and all question word segmentation vectors. The overall loss of the knowledge fragment is calculated as the relevance result based on the similarity.

[0066] The post-processing module is used to extract a set of sentences based on the relevance results and a preset relevance threshold, and send them together with the question to the prompt template to form the final input prompt. The input prompt is then input into the large language model to obtain the response result.

[0067] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

[0068] Corresponding to the aforementioned embodiment of a method for improving the quality of RAG technology-generated content based on semantic matching, the present invention also provides an embodiment of a device for improving the quality of RAG technology-generated content based on semantic matching.

[0069] See Figure 4 The present invention provides a generation apparatus for improving the quality of RAG technology based on semantic matching, comprising a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement a generation method for improving the quality of RAG technology based on semantic matching as described in the above embodiment.

[0070] The present invention provides an embodiment of a generation device for improving the quality of RAG technology-generated content based on semantic matching. This device can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 4 The diagram shown is a hardware structure diagram of any device with data processing capabilities, which is a generation device for improving the quality of RAG technology based on semantic matching, provided by the present invention. (Except for...) Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0071] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0072] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0073] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a method for improving the quality of RAG-generated content based on semantic matching, as described in the above embodiments.

[0074] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0075] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the aforementioned method for improving the quality of RAG-generated content based on semantic matching.

[0076] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0077] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for improving the quality of content generated by RAG technology based on semantic matching, characterized in that, The method includes: S1. The text data provided by the user is segmented into subtexts and input together with the question into the embedding machine to be converted into embedding vectors. The embedding vectors of the subtexts and the embedding vectors of the question are stored together in the vector database and knowledge fragments are filtered out. S2. Segment the questions and knowledge fragments into words, obtain question word segments and knowledge fragment word segments, and map them into vectors. Find the vector that matches the question vector in each knowledge fragment. Calculate the similarity between the matching word segment vectors, as well as the similarity between all knowledge fragment word segment vectors and all question word segment vectors. Calculate the overall loss of the knowledge fragments as the relevance result based on the similarity. S3. Extract the sentence set based on the relevance results and the preset relevance threshold, and send it along with the question to the prompt template to form the final input prompt. Input the input prompt into the large language model to obtain the response result.

2. The method for improving the quality of RAG technology-generated content based on semantic matching according to claim 1, characterized in that, The specific steps of filtering out knowledge fragments include: finding the j sub-text vectors that are closest to the question vector in the vector database, and using the knowledge fragments corresponding to these sub-text vectors as valid knowledge fragments provided by RAG.

3. The method for improving the quality of RAG technology-generated content based on semantic matching according to claim 1, characterized in that, The process involves segmenting the question and knowledge fragment into words, obtaining question-segmented words and knowledge fragment-segmented words, and mapping them into vectors. A word segmenter is then used to segment the words and map the question-segmented words and knowledge fragment-segmented words into vectors.

4. The method for improving the quality of RAG technology-generated content based on semantic matching according to claim 1, characterized in that, In calculating the similarity between matched word segmentation vectors, cosine similarity is used to calculate the similarity between two vectors.

5. The method for improving the quality of RAG technology-generated content based on semantic matching according to claim 1, characterized in that, The similarity between all knowledge segment word vectors and all question word vectors is calculated using cosine similarity.

6. The method for improving the quality of RAG technology-generated content based on semantic matching according to claim 1, characterized in that, The overall loss of knowledge fragments calculated through similarity, as a correlation result, includes: First, the similarity between matching word segmentation vectors and the similarity between knowledge fragment word segmentation vectors and question vector word segmentation are calculated. Then, the weighted loss is calculated using the temperature parameter, similarity, and the similarity weights between each word segmentation in the knowledge fragment and the question word segmentation. Finally, the weighted losses of all word segmentation pairs are averaged to obtain the overall loss of the knowledge segment.

7. The method for improving the quality of RAG technology-generated content based on semantic matching according to claim 1, characterized in that, The step of extracting a set of sentences based on relevance combination and a preset relevance threshold includes: The sentences are sorted from high to low scores based on the relevance results. Sentences with an overall loss higher than h are discarded based on a set relevance threshold h. The remaining set of sentences and the question are sent to the prompt template to form the final input prompt.

8. A system for implementing the method according to any one of claims 1-7, characterized in that, The system includes: a preprocessing module, the word segmenter, a relevance module, and a postprocessing module; The preprocessing module is used to segment the user-provided text data into subtexts and input them together with the question into the embedding machine to convert them into embedding vectors. The embedding vectors of the subtexts and the embedding vectors of the question are stored together in the vector database and knowledge fragments are filtered out. The word segmenter is used to segment the question and knowledge fragment into words, resulting in question word segments and knowledge fragment word segments; The relevance module is used to map question word segmentation and knowledge fragment word segmentation into vectors, find the vector that matches the question vector in each knowledge fragment, calculate the similarity between the matched word segmentation vectors, and the similarity between all knowledge fragment word segmentation vectors and all question word segmentation vectors. The overall loss of the knowledge fragment is calculated as the relevance result based on the similarity. The post-processing module is used to extract a set of sentences based on the relevance results and a preset relevance threshold, and send them together with the question to the prompt template to form the final input prompt. The input prompt is then input into the large language model to obtain the response result.

9. A generation apparatus for improving the quality of RAG (Rich Animation and Grading) technology-generated content based on semantic matching, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it implements a generation method for improving the quality of RAG technology-generated content based on semantic matching as described in any one of claims 1-7.

10. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements a generation method for improving the quality of RAG technology-generated content based on semantic matching as described in any one of claims 1-7.

Images