Application level retrieval enhancement generation strategy method and device

Abstract

The invention provides an application level retrieval enhancement generation strategy method and device. The method comprises the steps that 1, entity extraction is conducted on a query statement input by a user through a ChatGPT model; 2, performing fuzzy matching in a knowledge base according to the subject words; 3, combining all candidate knowledge to form a pre-retrieval result; 4, coding the pre-retrieval result and the query statement through a DPR word vector embedding model to generate a knowledge word vector and a query vector respectively; 5, calculating cosine similarity between the knowledge word vector and the query vector according to a similarity calculation method in a DPR retrieval method; and step 6, sorting the knowledge word vectors according to the cosine similarity. An external knowledge base is introduced, original questions and external knowledge are combined, more semantic information is added for model input, and therefore model performance and output result quality are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and device for generating an application-level retrieval enhancement strategy. Background Art

[0002] Large Language Model (LLM): refers to a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence.

[0003] DPR (Dense Passage Retrieval): Dense retrieval model, a practical tool in the field of natural language processing, can quickly find and locate relevant information in massive text data. The model uses a highly compressed vector representation to process large amounts of text to achieve efficient retrieval results. DPR is mainly used in question-answering systems, knowledge graph construction, search engines and other fields, and plays an important role in improving system efficiency and accuracy.

[0004] ChatGPT (Generative Pre-trained Transformer): is a powerful language generation model that can be trained to perform various natural language understanding tasks including entity extraction.

[0005] Prompt Engineering: refers to designing and optimizing the instructions input to artificial intelligence models (such as ChatGPT) to ensure that the model generates the expected output. By building accurate and detailed prompts, the model's understanding and responsiveness can be improved, thereby achieving efficient human-computer interaction.

[0006] Retrieval-augmented generation (RAG) refers to optimizing the output of large language models so that they can reference authoritative knowledge bases outside of the training data source before generating responses. However, existing retrieval-augmented generation methods may introduce irrelevant or false information, resulting in "hallucination" phenomena, so this technical problem needs to be solved urgently. Summary of the invention

[0007] In view of this, the present application provides an application-level retrieval enhancement strategy generation method and device to improve the ability to retrieve correct knowledge.

[0008] To achieve the above purpose, the technical solution of this application is: A method for generating an application-level retrieval enhancement strategy, comprising: Step 1: Use the ChatGPT model to perform entity extraction on the query statement input by the user to extract the subject words of the query statement; Step 2: Perform fuzzy matching in the knowledge base according to the subject words, and select the knowledge in the knowledge base that contains the subject words as candidate knowledge; Step 3: Merge all candidate knowledge to form pre-retrieval results; Step 4: Encode the pre-search results and the query statement respectively through the DPR word vector embedding model to generate a knowledge word vector and a query vector respectively; Step 5: Calculate the cosine similarity between the knowledge word vector and the query vector according to the similarity calculation method in the DPR retrieval method; Step 6: Sort the knowledge word vectors according to the cosine similarity, and select the first several knowledge word vectors with the highest similarity as retrieval result I.

[0009] Furthermore, the application-level search enhancement strategy generation method also includes: Step 7: Filter the search result I by ChatGPT to obtain the search result II; Step 8: Input the retrieval result II into a large language model for training, and use the output of the large language model as a candidate response to the original question; Step 9: Evaluate the candidate responses through ChatGPT, and select the best candidate response as the final output based on the evaluation results.

[0010] Furthermore, the specific formula for calculating the cosine similarity between the knowledge word vector and the query vector in step 5 is:

[0011] in, is the query vector, For the Knowledge word vector.

[0012] An application-level search enhancement strategy generation device, comprising: Extraction module: used to perform entity extraction on the query statement input by the user through the ChatGPT model, and extract the subject words of the query statement; Matching module: used to perform fuzzy matching in the knowledge base according to the subject words, and select the knowledge in the knowledge base containing the subject words as candidate knowledge; Merging module: used to merge all candidate knowledge to form pre-retrieval results; Encoding module: used for respectively encoding the pre-search result and the query statement through the DPR word vector embedding model to generate a knowledge word vector and a query vector respectively; Calculation module: used to calculate the cosine similarity between the knowledge word vector and the query vector according to the similarity calculation method in the DPR retrieval method; Sorting module: used to sort the knowledge word vectors according to the cosine similarity, and select the first several knowledge word vectors with the highest similarity as the retrieval result I.

[0013] Furthermore, the application-level search enhancement strategy generation device further includes: Filtering module: used to perform validity filtering on the search result I through ChatGPT to obtain the search result II; Training module: used for inputting the search result II into a large language model for training, and using the output of the large language model as a candidate response to the original question; Evaluation module: used to evaluate the candidate responses through ChatGPT, and select the best candidate response as the final output based on the evaluation results.

[0014] Furthermore, the calculation module performs calculations specifically according to the following formula:

[0015] in, is the query vector, For the Knowledge word vector.

[0016] The beneficial effects of this application are: 1. By introducing an external knowledge base, the original problem is combined with external knowledge, adding more semantic information to the model input, thereby improving the model performance and output quality; 2. In large-scale agricultural models based on knowledge constraints, the introduction of agricultural expertise through retrieval enhancement and the integration of this expertise into the retrieval strategy process can generate more accurate agricultural decisions; 3. By combining ChatGPT with the dense paragraph retrieval model, the ability to retrieve correct knowledge is improved; 4. The strategy selects the best responses generated, implementing knowledge constraints on large language models at the application level. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1A flowchart of a method for generating an application-level search enhancement strategy for this application; Figure 2 Another flow chart of a method for generating an application-level search enhancement strategy for the present application; Figure 3 This is a system architecture diagram of an application-level retrieval enhancement generation strategy for this application; Figure 4 This is a structural block diagram of an application-level retrieval enhancement generation strategy device of the present application; Figure 5 This is another structural block diagram of an application-level retrieval enhancement generation strategy device of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0020] like Figure 1 and 3 As shown, a method for generating an application-level retrieval enhancement strategy includes: Step 1: Use the ChatGPT model to perform entity extraction on the query statement input by the user to extract the subject words of the query statement; For the query statement or original question entered by the user, the GPT-4.0-Turbo model is used to extract entities from the query statement and extract a list of topic words. Entity extraction can optimize the effect of ChatGPT entity extraction through custom prompts.

[0021] For example, the specific prompt used is: "Extract the following paragraph that may be used as subject terms for fuzzy search in the database. If there are multiple terms, separate them with English commas ','".

[0022] Text: This is a thermosensitive three-line hybrid rice variety suitable for planting in central and northern Guangxi. The growth period of early rice is about 127 days, which is 2 days later than the control variety You838; the growth period of late rice is about 111 days, which is the same as You838. Answer: Which rice variety meets the requirements? Extraction results: Thermosensitive three-line hybrid rice, Central Guangxi, Northern Guangxi Text: Suitable for planting in the mid-late-maturing spring corn type area in Liaoning Province with an active accumulated temperature of more than 10℃ and more than 2800℃. Answer: What corn varieties meet the requirements? Extraction results: Liaoning Province ≥10℃ active accumulated temperature above 2800℃, mid-late maturing spring corn type area Text: The body is about 6.98mm long and 2mm wide. The back of the body is light yellow, gradually darkening to light brown towards the anal plate, and the ventral surface is light in color; the front half of the skull gradually darkens to black, and the front edge of the pronotum is slightly darker. The chest has 3 pairs of legs, and the legs are light brown. The abdomen has many horizontal folds, especially on the back; the body surface has rows of setae; the body becomes thicker and slightly curved before pupation. What disease may it have? Extraction results: The body is about 6.98mm long and 2mm wide. The back of the body is light yellow and light brown. The ventral surface is light in color and gradually becomes black. The front edge of the pronotum is slightly darker. There are 3 pairs of legs, which are light brown. The abdomen has many horizontal folds and rows of setae. The body becomes thicker before pupation. Step 2: Perform fuzzy matching in the knowledge base according to the subject words, and select the knowledge in the knowledge base that contains the subject words as candidate knowledge; The knowledge base refers to the agricultural knowledge base, and the knowledge sources of the agricultural knowledge base can be websites such as Baidu Encyclopedia, Seed Business Network, National Rice Center, and related agricultural professional books such as "China Agricultural Crop Diseases and Insect Pests". Through the above channels, information is collected and sorted, and finally an agricultural knowledge base containing information on multiple crops, tens of thousands of varieties, and hundreds of diseases and insect pests is formed.

[0023] Fuzzy matching is a commonly used method for data retrieval, that is, partial matching. For example, to accurately search for "agricultural knowledge base", the fuzzy query "*base*" means that all data with "base" can be retrieved.

[0024] Step 3: Merge all candidate knowledge to form pre-retrieval results; For example, based on the extracted four keywords "corn", "Liaoning", "mid-late maturity" and "10℃", four candidate knowledge sets containing each keyword are retrieved respectively, and the intersection of the four candidate knowledge sets is taken to form the pre-search result.

[0025] Step 4: Encode the pre-search results and the query statement through a DPR (Dense Paragraph Retrieval) word vector embedding model to generate a knowledge word vector and a query vector respectively; Step 5: Calculate the cosine similarity between the knowledge word vector and the query vector according to the similarity calculation method in the DPR retrieval method; The specific formula of cosine similarity is:

[0026] in, is the query vector, For the Knowledge word vector.

[0027] Step 6: Sort the knowledge word vectors according to the cosine similarity, and select the first several knowledge word vectors with the highest similarity as retrieval result I.

[0028] For example, you can select the top The knowledge word vector is taken as the retrieval result I.

[0029] like Figure 2 and 3 As shown, the application-level retrieval enhancement generation strategy method also includes: Step 7: Filter the search result I by ChatGPT to obtain the search result II; ChatGPT receives the knowledge items retrieved in the retrieval phase and analyzes them to evaluate their relevance and usefulness to the current query. Based on the evaluation results, ChatGPT will filter out useful knowledge items and filter out irrelevant or low-quality content in retrieval results I. The effect of ChatGPT in screening useful knowledge items can also be optimized by prompting engineering design specific prompts.

[0030] For example, the Prompt used for filtering is: "Judge whether the following context can answer the question. If the context contains the information required to answer the question, output '[Yes]', otherwise output '[No]'" Context: {Retrieve Knowledge} Question: {Original question} Answer: Yes.

[0031] Step 8: Input the retrieval result II into a large language model for training, and use the output of the large language model as a candidate response to the original question; The large language model may be GPT or ChatGPT, etc.

[0032] Step 9: Evaluate the candidate responses through ChatGPT, and select the best candidate response as the final output based on the evaluation results.

[0033] The generated responses are evaluated using ChatGPT, considering their fluency, relevance, and whether they accurately respond to the original question. Based on the evaluation results, ChatGPT selects the best response as the final output, ensuring that the answer the user receives is not only accurate in information, but also natural and easy to understand.

[0034] The prompt for selecting the best response used in this article is: Given a question and some search context and answers, the context is the search facts, and the answer number that can best answer the question is selected. There is only one best answer number.

[0035] Question: {Original question} Context {Order No. 1}: {Retrieved Knowledge 1} Answer {1}: {Candidate Response 1} … Context {Order number m}: {Retrieved knowledge m} Answer {m}: {Candidate response m} The best answer number is: m like Figure 4 As shown, an application-level retrieval enhancement generation strategy device includes: Extraction module 310: used to perform entity extraction on the query statement input by the user through the ChatGPT model, and extract the subject words of the query statement; For the query statement or original question entered by the user, the GPT-4.0-Turbo model is used to extract entities from the query statement and extract a list of topic words. Entity extraction can optimize the effect of ChatGPT entity extraction through custom prompts.

[0036] Matching module 320: used to perform fuzzy matching in the knowledge base according to the subject word, and select the knowledge in the knowledge base containing the subject word as candidate knowledge; The knowledge base refers to the agricultural knowledge base, and the knowledge sources of the agricultural knowledge base can be websites such as Baidu Encyclopedia, Seed Business Network, National Rice Center, and related agricultural professional books such as "China Agricultural Crop Diseases and Insect Pests". Through the above channels, information is collected and sorted, and finally an agricultural knowledge base containing information on multiple crops, tens of thousands of varieties, and hundreds of diseases and insect pests is formed.

[0037] Merging module 330: used to merge all candidate knowledge to form pre-search results; Encoding module 340: used to encode the pre-search result and the query statement respectively through the DPR word vector embedding model to generate a knowledge word vector and a query vector respectively; Calculation module 350: used to calculate the cosine similarity between the knowledge word vector and the query vector according to the similarity calculation method in the DPR retrieval method; The calculation module 350 specifically performs calculation according to the following formula:

[0038] in, is the query vector, For the Knowledge word vector.

[0039] Sorting module 360: used to sort the knowledge word vectors according to the cosine similarity, and select the first several knowledge word vectors with the highest similarity as retrieval result I.

[0040] like Figure 5 As shown, the application-level retrieval enhancement generation strategy device also includes: Filtering module 370: used to perform validity filtering on the search result I through ChatGPT to obtain search result II; Based on this, irrelevant or low-quality content in the search result I is filtered out. The effect of ChatGPT in screening useful knowledge items can also be optimized by prompting specific prompts for engineering design.

[0041] Training module 380: used for inputting the search result II into a large language model for training, and using the output of the large language model as a candidate answer to the original question; The large language model may be GPT or ChatGPT, etc.

[0042] Evaluation module 390: used to evaluate the candidate responses through ChatGPT, and select the best candidate response as the final output based on the evaluation results.

[0043] As for the device embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units, such as distributed on a server and a client. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0044] This application introduces an external knowledge base to combine the original question with external knowledge, adding more semantic information to the model input, thereby improving the model performance and output quality. In a large-scale agricultural model based on knowledge constraints, professional knowledge in the agricultural field is introduced through retrieval enhancement, and this professional knowledge is integrated into the retrieval strategy process, which can generate more accurate agricultural decisions. By combining ChatGPT with a dense paragraph retrieval model, the ability to retrieve correct knowledge is improved. The strategy selects the best responses generated, realizing knowledge constraints on large language models at the application level.

[0045] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating an application-level search enhancement strategy, characterized in that: include: Step 1: Use the ChatGPT model to perform entity extraction on the query statement input by the user to extract the subject words of the query statement; Step 2: Perform fuzzy matching in the knowledge base according to the subject words, and select the knowledge in the knowledge base that contains the subject words as candidate knowledge; Step 3: Merge all candidate knowledge to form pre-retrieval results; Step 4: Encode the pre-search results and the query statement respectively through the DPR word vector embedding model to generate a knowledge word vector and a query vector respectively; Step 5: Calculate the cosine similarity between the knowledge word vector and the query vector according to the similarity calculation method in the DPR retrieval method; Step 6: Sort the knowledge word vectors according to the cosine similarity, and select the first several knowledge word vectors with the highest similarity as retrieval result I.

2. The method for generating an application-level search enhancement strategy according to claim 1, characterized in that: The application-level search enhancement generation strategy method also includes: Step 7: Filter the search result I by ChatGPT to obtain the search result II; Step 8: Input the retrieval result II into a large language model for training, and use the output of the large language model as a candidate response to the original question; Step 9: Evaluate the candidate responses through ChatGPT, and select the best candidate response as the final output based on the evaluation results.

3. The method for generating an application-level search enhancement strategy according to claim 1 or 2, characterized in that: The specific formula for calculating the cosine similarity between the knowledge word vector and the query vector in step 5 is: in, is the query vector, For the Knowledge word vector.

4. An application-level search enhancement strategy generation device, characterized in that: include: Extraction module: used to perform entity extraction on the query statement input by the user through the ChatGPT model, and extract the subject words of the query statement; Matching module: used to perform fuzzy matching in the knowledge base according to the subject words, and select the knowledge in the knowledge base containing the subject words as candidate knowledge; Merging module: used to merge all candidate knowledge to form pre-retrieval results; Encoding module: used for respectively encoding the pre-search result and the query statement through the DPR word vector embedding model to generate a knowledge word vector and a query vector respectively; Calculation module: used to calculate the cosine similarity between the knowledge word vector and the query vector according to the similarity calculation method in the DPR retrieval method; Sorting module: used to sort the knowledge word vectors according to the cosine similarity, and select the first several knowledge word vectors with the highest similarity as the retrieval result I.

5. The device for generating an application-level search enhancement strategy as claimed in claim 4, characterized in that: The application-level search enhancement strategy generation device also includes: Filtering module: used to perform validity filtering on the search result I through ChatGPT to obtain the search result II; Training module: used for inputting the search result II into a large language model for training, and using the output of the large language model as a candidate response to the original question; Evaluation module: used to evaluate the candidate responses through ChatGPT, and select the best candidate response as the final output based on the evaluation results.

6. An application-level search enhancement strategy generation device as claimed in claim 4 or 5, characterized in that: The calculation module specifically performs calculation according to the following formula: in, is the query vector, For the Knowledge word vector.

Images