Intelligent policy question and answer method and system based on retrieval enhancement

Abstract

The invention provides an intelligent policy question and answer method and system based on retrieval enhancement, and the method comprises the steps: S1, enabling a question input and error correction module to record a question input by a user, and correcting wrong words in the question; s2, enabling a question and answer matching generation module to query or generate answers corresponding to the questions; s3, enabling an answer output and cache module to receive and output answers, and storing the questions and the corresponding answers into a question and answer cache library. Due to the fact that the user inputs the error correction rate more accurately, the content retrieval quality is improved. The knowledge graph query and vector database query result reordering are combined, so that the answer generation quality and the diversity of the generated result content are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent robot technology, and in particular to an intelligent policy question-answering method and system based on retrieval enhancement. Background Art

[0002] The existing general input error correction model cannot correct policy-related business sentences well. It lacks a vocabulary that integrates typos, homophones, synonyms, and confusion words, and combines it with the BERT word segmentation model trained on policy-related corpora. It uses the MacBERT Chinese natural language pre-training model as the basis for training and improving the policy sentence error correction model to improve the accuracy of error correction.

[0003] Currently, RAG's retrieval method is mainly based on text similarity. In order to speed up the retrieval, the ANN algorithm usually sacrifices some accuracy, resulting in a relatively random correlation between the top 2 to top 5 results and the query statement, which will affect the final generated results.

[0004] Patent document CN116610792B discloses an intelligent policy question-and-answer robot and method based on AI services, which relates to the field of intelligent robot technology. The policy question-and-answer robot of the invention includes a policy question-and-answer virtual robot and a policy question-and-answer virtual robot control system for controlling the work of the policy question-and-answer virtual robot. By analyzing the policy issues corresponding to the enterprise, the corresponding answer content of the enterprise is matched. If the policy issues corresponding to the enterprise cannot be analyzed, the related words of the enterprise's corresponding questions are screened out, and the enterprise is guided to make a selection, and then the answer scenario style is selected according to the type of enterprise to answer the policy issues. After the answer is completed, the corresponding experience effect of the enterprise is analyzed according to the enterprise's score. However, the invention does not combine the knowledge graph query and the vector database query result reordering. Summary of the invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide an intelligent policy question answering method and system based on retrieval enhancement.

[0006] According to the present invention, a retrieval-enhanced intelligent policy question-answering method is provided, comprising:

[0007] Step S1: Instruct the question input and error correction module to record the question input by the user and correct the wrong words in the question;

[0008] Step S2: Instruct the question-answer matching generation module to query or generate an answer corresponding to the question;

[0009] Step S3: Instruct the answer output and cache module to receive and output the answer, and store the question and the corresponding answer into the question and answer cache library.

[0010] Preferably, in step S1:

[0011] Question input and error correction module: It includes question input module and question error correction module. The question input module records the questions input by users, and the question error correction module corrects the typos, homophones, synonyms, and confusing words in the questions. The corrected questions are passed to the question-answer matching generation module. The question error correction part corrects the questions by analyzing the form and characteristics of the questions input by users.

[0012] The question correction module corrects errors in user questions through the question correction model, where the question correction model is trained by the MacBERT Chinese natural language pre-training model. The data set for training the question correction model is generated by policy documents. The policy documents are first split into segments according to periods, and the split segments are cleaned and deduplicated. Then, the BERT segmentation model trained based on policy-related corpus is used for word segmentation. The correct vocabulary obtained in this way is used as the output data of the training model. Then, the word library of wrong characters, homophones, synonyms, and confusion words is integrated. According to the statistical probability of wrong words entered by existing users, the correct words in the sample are replaced with the wrong words in the word library to obtain the input data of the training model.

[0013] The policy vocabulary recognition part uses a policy vocabulary model trained based on the RNN model to recognize and merge policy vocabulary for questions;

[0014] The document re-ranking part re-ranks the documents by searching the vector database.

[0015] Preferably, in step S2:

[0016] The question-answer matching generation module contains three query answer paths:

[0017] Path 1: Convert the corrected question into an embedded vector, and use the vector form to match whether there are similar historical questions in the question-answer cache. If the match is successful, pass the answer to the historical question as the answer to the current question to the answer cache and output module. If the match fails, use path 2 to query the answer.

[0018] Path 2: Segment the corrected question into words, identify and merge the proprietary policy vocabulary, match the question intent and query the knowledge graph. If the match is successful, pass the queried answer content to the answer cache and output module. If the match or query fails, continue to generate the answer using Path 3.

[0019] Path three: Query the similarity between the question and the policy document fragment based on the question embedding vector, select the top-K according to the similarity result for re-ranking, submit the re-ranked results and the prompt to the large language model, generate the final answer through the large language model and pass it to the answer cache and output module.

[0020] Preferably, the path 1 of the question-answer matching generation module includes a question embedding module and a cache matching module: the question embedding module converts the corrected question into a 768-dimensional embedding vector through the CoSENT sentence vector model, and the cache matching module matches the cached question in the question-answer cache library based on the question embedding vector; the question-answer cache library is established based on the FAISS vector database to store completed questions and answers; the matching process uses the cosine similarity formula to calculate the similarity between the current question and the cached question, and when the similarity is greater than the specified threshold, the match is successful, and the answer corresponding to the cached question is passed to the answer cache and output module as the answer to the current question; when it is less than the specified threshold, it is considered that the match fails, and the path 2 will be entered to retrieve the answer to the question;

[0021] Path 2 of the question-answer matching generation module specifically includes a question segmentation module and a knowledge graph query module: the question segmentation module uses the aforementioned trained BERT segmentation model to segment the corrected questions, and uses the proprietary policy vocabulary model trained based on the RNN model to identify and merge the segmented words with the proprietary policy vocabulary. The knowledge graph query module matches the identified and merged question vocabulary with the predefined question intent. After the match is successful, the answer-related content is searched in the knowledge graph established based on Neo4J. If the query is successful, the query result is passed to the answer cache and output module; if the match or query fails, enter path 3 to generate the question answer;

[0022] Path three of the question-answer matching generation module specifically includes a document fragment query and re-ranking module and a large language model answer generation module: the document fragment query and re-ranking module calls the question embedding vector, matches the content in the main vector database and re-ranks it; the main vector database is established based on pgvector, storing split and cleaned policy document fragments; the matching process uses the cosine similarity formula to calculate the similarity between the question and the policy document fragment; based on the similarity calculation results, the module takes the 30 fragments with the highest similarity and returns them, uses the cross-coding model BGE Reranker to re-rank the 30 results with the highest similarity, and then takes the top 5 re-ranked results and passes them to the large language model answer generation module. In the large language model answer generation module, the 5 policy document fragments with the highest similarity and the preset prompt are submitted to the large language model, and the final answer is generated by the large language model and passed to the answer cache and output module.

[0023] Preferably, in step S3:

[0024] The answer output and cache module includes an answer output module and a question and answer cache module: when the answer output module receives the answer delivered by path one of the question and answer matching generation module, it directly outputs the answer to the user; when the answer output module receives the answer content delivered by path two of the question and answer matching generation module, it fills the content into a preset answer template and then outputs it to the user, and the question and answer cache module saves the question and answer to the question and answer cache library; when the answer output module receives the answer delivered by path three of the question and answer matching generation module, it outputs the answer to the user and the question and answer cache module saves the question and answer to the question and answer cache library.

[0025] According to the present invention, an intelligent policy question-answering system based on retrieval enhancement is provided, comprising:

[0026] Question input and error correction module: records the questions input by users and corrects the wrong words in the questions;

[0027] Question-answer matching generation module: query or generate answers corresponding to questions;

[0028] Answer output and cache module: receives and outputs answers, and stores questions and corresponding answers in the question and answer cache.

[0029] Preferably, in the question input and error correction module:

[0030] It includes a question input module and a question correction module. The question input module records the questions input by the user, and the question correction module corrects the typos, homophones, synonyms, and confusions in the questions. The corrected questions are passed to the question-answer matching generation module. The question correction part corrects the questions by analyzing the form and features of the questions input by the user.

[0031] The question correction module corrects errors in user questions through the question correction model, where the question correction model is trained by the MacBERT Chinese natural language pre-training model. The data set for training the question correction model is generated by policy documents. The policy documents are first split into segments according to periods, and the split segments are cleaned and deduplicated. Then, the BERT segmentation model trained based on policy-related corpus is used for word segmentation. The correct vocabulary obtained in this way is used as the output data of the training model. Then, the word library of wrong characters, homophones, synonyms, and confusion words is integrated. According to the statistical probability of wrong words entered by existing users, the correct words in the sample are replaced with the wrong words in the word library to obtain the input data of the training model.

[0032] The policy vocabulary recognition part uses a policy vocabulary model trained based on the RNN model to recognize and merge policy vocabulary for questions;

[0033] The document re-ranking part re-ranks the documents by searching the vector database.

[0034] Preferably, in the question-answer matching generation module:

[0035] The question-answer matching generation module contains three query answer paths:

[0036] Path 1: Convert the corrected question into an embedded vector, and use the vector form to match whether there are similar historical questions in the question-answer cache. If the match is successful, pass the answer to the historical question as the answer to the current question to the answer cache and output module. If the match fails, use path 2 to query the answer.

[0037] Path 2: Segment the corrected question into words, identify and merge the proprietary policy vocabulary, match the question intent and query the knowledge graph. If the match is successful, pass the queried answer content to the answer cache and output module. If the match or query fails, continue to generate the answer using Path 3.

[0038] Path three: Query the similarity between the question and the policy document fragment based on the question embedding vector, select the top-K according to the similarity result for re-ranking, submit the re-ranked results and the prompt to the large language model, generate the final answer through the large language model and pass it to the answer cache and output module.

[0039] Preferably, the path 1 of the question-answer matching generation module includes a question embedding module and a cache matching module: the question embedding module converts the corrected question into a 768-dimensional embedding vector through the CoSENT sentence vector model, and the cache matching module matches the cached question in the question-answer cache library based on the question embedding vector; the question-answer cache library is established based on the FAISS vector database to store completed questions and answers; the matching process uses the cosine similarity formula to calculate the similarity between the current question and the cached question, and when the similarity is greater than the specified threshold, the match is successful, and the answer corresponding to the cached question is passed to the answer cache and output module as the answer to the current question; when it is less than the specified threshold, it is considered that the match fails, and the path 2 will be entered to retrieve the answer to the question;

[0040] Path 2 of the question-answer matching generation module specifically includes a question segmentation module and a knowledge graph query module: the question segmentation module uses the aforementioned trained BERT segmentation model to segment the corrected questions, and uses the proprietary policy vocabulary model trained based on the RNN model to identify and merge the segmented words with the proprietary policy vocabulary. The knowledge graph query module matches the identified and merged question vocabulary with the predefined question intent. After the match is successful, the answer-related content is searched in the knowledge graph established based on Neo4J. If the query is successful, the query result is passed to the answer cache and output module; if the match or query fails, enter path 3 to generate the question answer;

[0041] Path three of the question-answer matching generation module specifically includes a document fragment query and re-ranking module and a large language model answer generation module: the document fragment query and re-ranking module calls the question embedding vector, matches the content in the main vector database and re-ranks it; the main vector database is established based on pgvector, storing split and cleaned policy document fragments; the matching process uses the cosine similarity formula to calculate the similarity between the question and the policy document fragment; based on the similarity calculation results, the module takes the 30 fragments with the highest similarity and returns them, uses the cross-coding model BGE Reranker to re-rank the 30 results with the highest similarity, and then takes the top 5 re-ranked results and passes them to the large language model answer generation module. In the large language model answer generation module, the 5 policy document fragments with the highest similarity and the preset prompt are submitted to the large language model, and the final answer is generated by the large language model and passed to the answer cache and output module.

[0042] Preferably, in the answer output and cache module:

[0043] The answer output and cache module includes an answer output module and a question and answer cache module: when the answer output module receives the answer delivered by path one of the question and answer matching generation module, it directly outputs the answer to the user; when the answer output module receives the answer content delivered by path two of the question and answer matching generation module, it fills the content into a preset answer template and then outputs it to the user, and the question and answer cache module saves the question and answer to the question and answer cache library; when the answer output module receives the answer delivered by path three of the question and answer matching generation module, it outputs the answer to the user and the question and answer cache module saves the question and answer to the question and answer cache library.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The present invention improves the quality of content retrieval. The current retrieval method of RAG is mainly based on text similarity, but this cannot fully guarantee the high relevance of the retrieved content with the query statement;

[0046] 2. The present invention improves the quality of answer generation. Currently, the generation method of RAG is mainly based on language models, but language models have certain limitations. Relying only on language models may generate inaccurate or unsmooth texts.

[0047] 3. The present invention improves the content retrieval quality due to a more accurate user input error correction rate. The quality of answer generation and the diversity of generated result content are improved due to the combination of knowledge graph query and vector database query result reordering. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0049] Figure 1 It is the system module block diagram;

[0050] Figure 2 It is a work flow chart;

[0051] Figure 3 This is the system architecture diagram. DETAILED DESCRIPTION

[0052] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0053] Embodiment 1:

[0054] The present invention integrates lexicons such as typos, homophones, synonyms, and confusions, and combines a BERT word segmentation model trained for policy corpora. The MacBERT Chinese natural language pre-training model is used as the basis for training and improving the policy sentence error correction model to improve the accuracy of error correction. The erroneous input in the training data set is obtained by replacing the correct words in the sample with the probability of the existing user input erroneous word statistical model.

[0055] This technical solution returns more query results by increasing the size of the retrieval top_k, and then re-ranks the retrieval results, calculating the scores one by one in a small range, so that the generated text is more likely to be related to the retrieval results, thereby improving the generation quality.

[0056] According to an intelligent policy question answering method based on retrieval enhancement provided by the present invention, Figure 1-Figure 3 As shown, including:

[0057] Step S1: Instruct the question input and error correction module to record the question input by the user and correct the wrong words in the question;

[0058] Specifically, in step S1:

[0059] Question input and error correction module: It includes question input module and question error correction module. The question input module records the questions input by users, and the question error correction module corrects the typos, homophones, synonyms, and confusing words in the questions. The corrected questions are passed to the question-answer matching generation module. The question error correction part corrects the questions by analyzing the form and characteristics of the questions input by users.

[0060] The question correction module corrects errors in user questions through the question correction model, where the question correction model is trained by the MacBERT Chinese natural language pre-training model. The data set for training the question correction model is generated by policy documents. The policy documents are first split into segments according to periods, and the split segments are cleaned and deduplicated. Then, the BERT segmentation model trained based on policy-related corpus is used for word segmentation. The correct vocabulary obtained in this way is used as the output data of the training model. Then, the word library of wrong characters, homophones, synonyms, and confusion words is integrated. According to the statistical probability of wrong words entered by existing users, the correct words in the sample are replaced with the wrong words in the word library to obtain the input data of the training model.

[0061] The policy vocabulary recognition part uses a policy vocabulary model trained based on the RNN model to recognize and merge policy vocabulary for questions;

[0062] The document re-ranking part re-ranks the documents by searching the vector database.

[0063] Step S2: Instruct the question-answer matching generation module to query or generate an answer corresponding to the question;

[0064] Specifically, in step S2:

[0065] The question-answer matching generation module contains three query answer paths:

[0066] Path 1: Convert the corrected question into an embedded vector, and use the vector form to match whether there are similar historical questions in the question-answer cache. If the match is successful, pass the answer to the historical question as the answer to the current question to the answer cache and output module. If the match fails, use path 2 to query the answer.

[0067] Path 2: Segment the corrected question into words, identify and merge the proprietary policy vocabulary, match the question intent and query the knowledge graph. If the match is successful, pass the queried answer content to the answer cache and output module. If the match or query fails, continue to generate the answer using Path 3.

[0068] Path three: Query the similarity between the question and the policy document fragment based on the question embedding vector, select the top-K according to the similarity result for re-ranking, submit the re-ranked results and the prompt to the large language model, generate the final answer through the large language model and pass it to the answer cache and output module.

[0069] Specifically, the path 1 of the question-answer matching generation module includes a question embedding module and a cache matching module: the question embedding module converts the corrected question into a 768-dimensional embedding vector through the CoSENT sentence vector model, and the cache matching module matches the cached question in the question-answer cache based on the question embedding vector; the question-answer cache is established based on the FAISS vector database to store completed questions and answers; the matching process uses the cosine similarity formula to calculate the similarity between the current question and the cached question. When the similarity is greater than the specified threshold, the match is successful, and the answer corresponding to the cached question is passed to the answer cache and output module as the answer to the current question; when it is less than the specified threshold, it is considered that the match fails, and the answer to the question will be retrieved in path 2;

[0070] Path 2 of the question-answer matching generation module specifically includes a question segmentation module and a knowledge graph query module: the question segmentation module uses the aforementioned trained BERT segmentation model to segment the corrected questions, and uses the proprietary policy vocabulary model trained based on the RNN model to identify and merge the segmented words with the proprietary policy vocabulary. The knowledge graph query module matches the identified and merged question vocabulary with the predefined question intent. After the match is successful, the answer-related content is searched in the knowledge graph established based on Neo4J. If the query is successful, the query result is passed to the answer cache and output module; if the match or query fails, enter path 3 to generate the question answer;

[0071] Path three of the question-answer matching generation module specifically includes a document fragment query and re-ranking module and a large language model answer generation module: the document fragment query and re-ranking module calls the question embedding vector, matches the content in the main vector database and re-ranks it; the main vector database is established based on pgvector, storing split and cleaned policy document fragments; the matching process uses the cosine similarity formula to calculate the similarity between the question and the policy document fragment; based on the similarity calculation results, the module takes the 30 fragments with the highest similarity and returns them, uses the cross-coding model BGE Reranker to re-rank the 30 results with the highest similarity, and then takes the top 5 re-ranked results and passes them to the large language model answer generation module. In the large language model answer generation module, the 5 policy document fragments with the highest similarity and the preset prompt are submitted to the large language model, and the final answer is generated by the large language model and passed to the answer cache and output module.

[0072] Step S3: Instruct the answer output and cache module to receive and output the answer, and store the question and the corresponding answer into the question and answer cache library.

[0073] Specifically, in step S3:

[0074] The answer output and cache module includes an answer output module and a question and answer cache module: when the answer output module receives the answer delivered by path one of the question and answer matching generation module, it directly outputs the answer to the user; when the answer output module receives the answer content delivered by path two of the question and answer matching generation module, it fills the content into a preset answer template and then outputs it to the user, and the question and answer cache module saves the question and answer to the question and answer cache library; when the answer output module receives the answer delivered by path three of the question and answer matching generation module, it outputs the answer to the user and the question and answer cache module saves the question and answer to the question and answer cache library.

[0075] Embodiment 2:

[0076] Embodiment 2 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0077] The present invention also provides a retrieval-enhanced intelligent policy question-answering system, which can be implemented by executing the process steps of the retrieval-enhanced intelligent policy question-answering method, that is, those skilled in the art can understand the retrieval-enhanced intelligent policy question-answering method as a preferred implementation of the retrieval-enhanced intelligent policy question-answering system.

[0078] According to the present invention, an intelligent policy question-answering system based on retrieval enhancement is provided, comprising:

[0079] Question input and error correction module: records the questions input by users and corrects the wrong words in the questions;

[0080] Specifically, in the problem input and error correction module:

[0081] It includes a question input module and a question correction module. The question input module records the questions input by the user, and the question correction module corrects the typos, homophones, synonyms, and confusions in the questions. The corrected questions are passed to the question-answer matching generation module. The question correction part corrects the questions by analyzing the form and features of the questions input by the user.

[0082] The question correction module corrects errors in user questions through the question correction model, where the question correction model is trained by the MacBERT Chinese natural language pre-training model. The data set for training the question correction model is generated by policy documents. The policy documents are first split into segments according to periods, and the split segments are cleaned and deduplicated. Then, the BERT segmentation model trained based on policy-related corpus is used for word segmentation. The correct vocabulary obtained in this way is used as the output data of the training model. Then, the word library of wrong characters, homophones, synonyms, and confusion words is integrated. According to the statistical probability of wrong words entered by existing users, the correct words in the sample are replaced with the wrong words in the word library to obtain the input data of the training model.

[0083] The policy vocabulary recognition part uses a policy vocabulary model trained based on the RNN model to recognize and merge policy vocabulary for questions;

[0084] The document re-ranking part re-ranks the documents by searching the vector database.

[0085] Question-answer matching generation module: query or generate answers corresponding to questions;

[0086] Specifically, in the question-answer matching generation module:

[0087] The question-answer matching generation module contains three query answer paths:

[0088] Path 1: Convert the corrected question into an embedded vector, and use the vector form to match whether there are similar historical questions in the question-answer cache. If the match is successful, pass the answer to the historical question as the answer to the current question to the answer cache and output module. If the match fails, use path 2 to query the answer.

[0089] Path 2: Segment the corrected question into words, identify and merge the proprietary policy vocabulary, match the question intent and query the knowledge graph. If the match is successful, pass the queried answer content to the answer cache and output module. If the match or query fails, continue to generate the answer using Path 3.

[0090] Path three: Query the similarity between the question and the policy document fragment based on the question embedding vector, select the top-K according to the similarity result for re-ranking, submit the re-ranked results and the prompt to the large language model, generate the final answer through the large language model and pass it to the answer cache and output module.

[0091] Specifically, the path 1 of the question-answer matching generation module includes a question embedding module and a cache matching module: the question embedding module converts the corrected question into a 768-dimensional embedding vector through the CoSENT sentence vector model, and the cache matching module matches the cached question in the question-answer cache based on the question embedding vector; the question-answer cache is established based on the FAISS vector database to store completed questions and answers; the matching process uses the cosine similarity formula to calculate the similarity between the current question and the cached question. When the similarity is greater than the specified threshold, the match is successful, and the answer corresponding to the cached question is passed to the answer cache and output module as the answer to the current question; when it is less than the specified threshold, it is considered that the match fails, and the answer to the question will be retrieved in path 2;

[0092] Path 2 of the question-answer matching generation module specifically includes a question segmentation module and a knowledge graph query module: the question segmentation module uses the aforementioned trained BERT segmentation model to segment the corrected questions, and uses the proprietary policy vocabulary model trained based on the RNN model to identify and merge the segmented words with the proprietary policy vocabulary. The knowledge graph query module matches the identified and merged question vocabulary with the predefined question intent. After the match is successful, the answer-related content is searched in the knowledge graph established based on Neo4J. If the query is successful, the query result is passed to the answer cache and output module; if the match or query fails, enter path 3 to generate the question answer;

[0093] Path three of the question-answer matching generation module specifically includes a document fragment query and re-ranking module and a large language model answer generation module: the document fragment query and re-ranking module calls the question embedding vector, matches the content in the main vector database and re-ranks it; the main vector database is established based on pgvector, storing split and cleaned policy document fragments; the matching process uses the cosine similarity formula to calculate the similarity between the question and the policy document fragment; based on the similarity calculation results, the module takes the 30 fragments with the highest similarity and returns them, uses the cross-coding model BGE Reranker to re-rank the 30 results with the highest similarity, and then takes the top 5 re-ranked results and passes them to the large language model answer generation module. In the large language model answer generation module, the 5 policy document fragments with the highest similarity and the preset prompt are submitted to the large language model, and the final answer is generated by the large language model and passed to the answer cache and output module.

[0094] Answer output and cache module: receives and outputs answers, and stores questions and corresponding answers in the question and answer cache.

[0095] Specifically, in the answer output and cache module:

[0096] The answer output and cache module includes an answer output module and a question and answer cache module: when the answer output module receives the answer delivered by path one of the question and answer matching generation module, it directly outputs the answer to the user; when the answer output module receives the answer content delivered by path two of the question and answer matching generation module, it fills the content into a preset answer template and then outputs it to the user, and the question and answer cache module saves the question and answer to the question and answer cache library; when the answer output module receives the answer delivered by path three of the question and answer matching generation module, it outputs the answer to the user and the question and answer cache module saves the question and answer to the question and answer cache library.

[0097] Embodiment 3:

[0098] Embodiment 3 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0099] An intelligent policy question-answering system based on retrieval enhancement includes three modules: question input and error correction, question-answer matching generation, and answer output and cache. The question input and error correction module is used to record the questions input by the user and correct the wrong words in the questions; the question-answer matching generation module is used to query or generate the answers corresponding to the questions; the answer output and cache module is used to receive and output the answers, and store the questions and the corresponding answers in the question-answer cache.

[0100] The question correction part automatically performs error correction by analyzing the form and characteristics of the questions entered by the user to ensure the correctness and completeness of the questions, and then provides search results that better meet the user's needs.

[0101] In the policy vocabulary recognition part, a proprietary policy vocabulary model trained based on the RNN model is used to identify and merge policy vocabulary for questions, thereby improving the relevance and accuracy of downstream tasks.

[0102] In the document re-ranking part, by re-ranking the documents of the vector database search results, the relevance of the generated results to the search questions is improved, and the accuracy, diversity and pertinence of the answer content are improved.

[0103] The question input and error correction module includes a question input module and a question error correction module. The question input module is used to record the questions input by the user, and the question error correction module is used to correct typos, homophones, synonyms, confusing words and other errors in the questions. The corrected questions are passed to the question-answer matching generation module.

[0104] The question-answer matching generation module contains three paths for querying answers. First, path one converts the corrected question into an embedding vector, and matches whether there are similar historical questions in the question-answer cache in the form of a vector. If the match is successful, the answer corresponding to the historical question is passed to the answer cache and output module as the answer to the current question. If the match fails, path two is used to query the answer. Path two tokenizes the corrected question, then identifies and merges the proprietary policy vocabulary, and finally performs question intent matching and knowledge graph query. If the match is successful and the query is successful, the queried answer content is passed to the answer cache and output module. If the match or query fails, path three is used to generate the answer. Path three queries the similarity between the question and the policy document fragment based on the question embedding vector, selects the top-K for re-ranking based on the similarity result, and submits the re-ranked result and the prompt to the large language model (LLM). The final answer is generated by the model and passed to the answer cache and output module.

[0105] The answer output and cache module includes an answer output module and a question and answer cache module. When the answer output module receives the answer delivered by path one, it directly outputs the answer to the user. When the answer output module receives the answer content delivered by path two, it fills the content into the preset answer template and then outputs it to the user. At the same time, the question and answer cache module saves the question and answer to the question and answer cache library. When the answer output module receives the answer delivered by path three, it outputs the answer to the user and the question and answer cache module saves the question and answer to the question and answer cache library.

[0106] The question correction module corrects errors in user questions through the question correction model, where the question correction model is trained by the MacBERT Chinese natural language pre-training model. The data set for training the question correction model is generated by policy documents. The policy documents are first split into segments according to periods, and the split segments are cleaned and deduplicated. Then, the BERT segmentation model trained based on policy-related corpus is used for word segmentation. The correct vocabulary obtained is used as the output data of the training model. Then, the word library of typos, homophonic wrong words, synonymous wrong words, and confusing wrong words is integrated. According to the statistical probability of incorrect words entered by existing users, the correct words in the sample are replaced with the incorrect words in the word library to obtain the input data of the training model.

[0107] Path 1 of the question-answer matching generation module specifically includes two sub-modules, namely the question embedding module and the cache matching module. The question embedding module converts the corrected question into a 768-dimensional embedding vector through the CoSENT (Cosine Sentence) sentence vector model. The cache matching module matches similar cached questions in the question-answer cache based on the question embedding vector. Specifically, the question-answer cache is established based on the FAISS vector database to store completed questions and answers. The matching process uses the cosine similarity formula to calculate the similarity between the current question and the cached question. When the similarity is greater than the specified threshold, the match is considered successful, and the answer corresponding to the cached question is regarded as the answer to the current question and passed to the answer cache and output module; when it is less than the specified threshold, the match is considered to have failed, and path 2 will be entered to retrieve the answer to the question.

[0108] Path 2 of the question-answer matching generation module specifically includes two submodules, namely the question segmentation module and the knowledge graph query module. The question segmentation module uses the aforementioned trained BERT segmentation model to segment the corrected questions, and then uses the proprietary policy vocabulary model trained based on the RNN model to identify and merge the proprietary policy vocabulary. The knowledge graph query module first matches the identified and merged question vocabulary with the predefined question intent. After the match is successful, it queries the answer-related content in the knowledge graph established based on Neo4J. If the query is successful, the query result is passed to the answer cache and output module. If the match or query fails, enter path 3 to generate the question answer.

[0109] Path three of the question-answer matching generation module specifically includes two submodules, namely the document fragment query and re-ranking module and the LLM answer generation module. The document fragment query and re-ranking module calls the question embedding vector, matches and re-ranks similar content in the main vector database. Specifically, the main vector database is established based on pgvector and stores the split and cleaned policy document fragments. The matching process uses the cosine similarity formula to calculate the similarity between the question and the policy document fragment. According to the similarity calculation results, the module takes the top-30 similarity fragments and returns them, and then uses the cross-encoding model BGEReranker to re-rank the top-30 results, and then takes the top-5 of the re-ranking results and passes them to the LLM answer generation module to increase the result hit rate. In the LLM answer generation module, the policy document fragments with the top-5 similarity and the preset prompt are submitted to the LLM, and the final answer is generated by the LLM and passed to the answer cache and output module.

[0110] like Figure 3 As shown, according to the above technical solution, an intelligent policy question-answering system based on retrieval enhancement is built, including an external access layer, a core module layer, an algorithm layer, a platform layer and a basic resource layer.

[0111] The external access layer is responsible for interacting with users and providing interfaces for users to access the system, including web portals, WeChat applets, API SDKs, APPs, and public components.

[0112] The core module layer is responsible for processing user requests, including question correction, question and answer caching, policy vocabulary recognition, template filling, reordering, etc.

[0113] The algorithm layer provides core algorithm support, including PyTorch, MacBERT, LLM, RNN and other algorithms and models.

[0114] The platform layer provides the basic services required for system operation, including vector databases, containers, computing platforms, etc.

[0115] The basic resource layer provides the hardware resources required for system operation, including servers, storage, networks, etc.

[0116] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0117] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A retrieval-enhanced intelligent policy question answering method, characterized in that: include: Step S1: Instruct the question input and error correction module to record the question input by the user and correct the wrong words in the question; Step S2: Instruct the question-answer matching generation module to query or generate an answer corresponding to the question; Step S3: Instruct the answer output and cache module to receive and output the answer, and store the question and the corresponding answer into the question and answer cache library.

2. The intelligent policy question answering method based on retrieval enhancement according to claim 1 is characterized in that: In step S1: Question input and error correction module: It includes question input module and question error correction module. The question input module records the questions input by the user. The question error correction module corrects the typos, homophones, synonyms, and confusing words in the questions. The corrected questions are passed to the question-answer matching generation module. The question correction part analyzes the form and characteristics of the user input questions and performs error correction; The question correction module corrects errors in user questions through the question correction model, where the question correction model is trained by the MacBERT Chinese natural language pre-training model. The data set for training the question correction model is generated by policy documents. The policy documents are first split into segments according to periods, and the split segments are cleaned and deduplicated. Then, the BERT segmentation model trained based on policy-related corpus is used for word segmentation. The correct vocabulary obtained in this way is used as the output data of the training model. Then, the word library of wrong characters, homophones, synonyms, and confusion words is integrated. According to the statistical probability of wrong words entered by existing users, the correct words in the sample are replaced with the wrong words in the word library to obtain the input data of the training model. The policy vocabulary recognition part uses a policy vocabulary model trained based on the RNN model to recognize and merge policy vocabulary for questions; The document re-ranking part re-ranks the documents by searching the vector database.

3. The intelligent policy question answering method based on retrieval enhancement according to claim 1 is characterized in that: In step S2: The question-answer matching generation module contains three query answer paths: Path 1: Convert the corrected question into an embedded vector, and use the vector form to match whether there are similar historical questions in the question-answer cache. If the match is successful, pass the answer to the historical question as the answer to the current question to the answer cache and output module. If the match fails, use path 2 to query the answer. Path 2: Segment the corrected question into words, identify and merge the proprietary policy vocabulary, match the question intent and query the knowledge graph. If the match is successful, pass the queried answer content to the answer cache and output module. If the match or query fails, continue to generate the answer using Path 3. Path three: Query the similarity between the question and the policy document fragment based on the question embedding vector, select the top-K according to the similarity result for re-ranking, submit the re-ranked results and the prompt to the large language model, generate the final answer through the large language model and pass it to the answer cache and output module.

4. The intelligent policy question answering method based on retrieval enhancement according to claim 3 is characterized in that: Path 1 of the question-answer matching generation module includes a question embedding module and a cache matching module: the question embedding module converts the corrected question into a 768-dimensional embedding vector through the CoSENT sentence vector model, and the cache matching module matches the cached question in the question-answer cache based on the question embedding vector; the question-answer cache is established based on the FAISS vector database to store completed questions and answers; The matching process uses the cosine similarity formula to calculate the similarity between the current question and the cached question. When the similarity is greater than the specified threshold, the match is successful, and the answer corresponding to the cached question is passed to the answer cache and output module as the answer to the current question; when it is less than the specified threshold, the match is considered to have failed, and the second path will be entered to retrieve the answer to the question; Path 2 of the question-answer matching generation module specifically includes a question segmentation module and a knowledge graph query module: the question segmentation module uses the aforementioned trained BERT segmentation model to segment the corrected questions, and uses the proprietary policy vocabulary model trained based on the RNN model to identify and merge the segmented words with the proprietary policy vocabulary. The knowledge graph query module matches the identified and merged question vocabulary with the predefined question intent. After the match is successful, it queries the answer-related content in the knowledge graph established based on Neo4J. If the query is successful, the query result is passed to the answer cache and output module. If the match or query fails, go to path three to generate the answer to the question; Path three of the question-answer matching generation module specifically includes a document fragment query and re-ranking module and a large language model answer generation module: the document fragment query and re-ranking module calls the question embedding vector, matches the content in the main vector database and re-ranks it; the main vector database is established based on pgvector, storing split and cleaned policy document fragments; the matching process uses the cosine similarity formula to calculate the similarity between the question and the policy document fragment; based on the similarity calculation results, the module takes the 30 fragments with the highest similarity and returns them, uses the cross-coding model BGE Reranker to re-rank the 30 results with the highest similarity, and then takes the top 5 re-ranked results and passes them to the large language model answer generation module. In the large language model answer generation module, the 5 policy document fragments with the highest similarity and the preset prompt are submitted to the large language model, and the final answer is generated by the large language model and passed to the answer cache and output module.

5. The intelligent policy question answering method based on retrieval enhancement according to claim 1 is characterized in that: In step S3: The answer output and cache module includes an answer output module and a question and answer cache module: when the answer output module receives the answer delivered by path one of the question and answer matching generation module, it directly outputs the answer to the user; when the answer output module receives the answer content delivered by path two of the question and answer matching generation module, it fills the content into a preset answer template and then outputs it to the user, and the question and answer cache module saves the question and answer to the question and answer cache library; when the answer output module receives the answer delivered by path three of the question and answer matching generation module, it outputs the answer to the user and the question and answer cache module saves the question and answer to the question and answer cache library.

6. An intelligent policy question answering system based on retrieval enhancement, characterized in that: include: Question input and error correction module: records the questions input by users and corrects the wrong words in the questions; Question-answer matching generation module: query or generate answers corresponding to questions; Answer output and cache module: receives and outputs answers, and stores questions and corresponding answers in the question and answer cache.

7. The intelligent policy question answering system based on retrieval enhancement according to claim 6 is characterized in that: In the problem input and error correction module: It includes a question input module and a question correction module. The question input module records the questions input by the user. The question correction module corrects the typos, homophones, synonyms, and confusing words in the questions. The corrected questions are passed to the question-answer matching generation module. The question correction part analyzes the form and characteristics of the user input questions and performs error correction; The question correction module corrects errors in user questions through the question correction model, where the question correction model is trained by the MacBERT Chinese natural language pre-training model. The data set for training the question correction model is generated by policy documents. The policy documents are first split into segments according to periods, and the split segments are cleaned and deduplicated. Then, the BERT segmentation model trained based on policy-related corpus is used for word segmentation. The correct vocabulary obtained in this way is used as the output data of the training model. Then, the word library of wrong characters, homophones, synonyms, and confusion words is integrated. According to the statistical probability of wrong words entered by existing users, the correct words in the sample are replaced with the wrong words in the word library to obtain the input data of the training model. The policy vocabulary recognition part uses a policy vocabulary model trained based on the RNN model to recognize and merge policy vocabulary for questions; The document re-ranking part re-ranks the documents by searching the vector database.

8. The intelligent policy question answering system based on retrieval enhancement according to claim 6 is characterized in that: In the question-answer matching generation module: The question-answer matching generation module contains three query answer paths: Path 1: Convert the corrected question into an embedded vector, and use the vector form to match whether there are similar historical questions in the question-answer cache. If the match is successful, pass the answer to the historical question as the answer to the current question to the answer cache and output module. If the match fails, use path 2 to query the answer. Path 2: Segment the corrected question into words, identify and merge the proprietary policy vocabulary, match the question intent and query the knowledge graph. If the match is successful, pass the queried answer content to the answer cache and output module. If the match or query fails, continue to generate the answer using Path 3. Path three: Query the similarity between the question and the policy document fragment based on the question embedding vector, select the top-K according to the similarity result for re-ranking, submit the re-ranked results and the prompt to the large language model, generate the final answer through the large language model and pass it to the answer cache and output module.

9. The intelligent policy question answering system based on retrieval enhancement according to claim 8, characterized in that: Path 1 of the question-answer matching generation module includes a question embedding module and a cache matching module: the question embedding module converts the corrected question into a 768-dimensional embedding vector through the CoSENT sentence vector model, and the cache matching module matches the cached question in the question-answer cache based on the question embedding vector; the question-answer cache is established based on the FAISS vector database to store completed questions and answers; The matching process uses the cosine similarity formula to calculate the similarity between the current question and the cached question. When the similarity is greater than the specified threshold, the match is successful, and the answer corresponding to the cached question is passed to the answer cache and output module as the answer to the current question; when it is less than the specified threshold, the match is considered to have failed, and the second path will be entered to retrieve the answer to the question; Path 2 of the question-answer matching generation module specifically includes a question segmentation module and a knowledge graph query module: the question segmentation module uses the aforementioned trained BERT segmentation model to segment the corrected questions, and uses the proprietary policy vocabulary model trained based on the RNN model to identify and merge the segmented words with the proprietary policy vocabulary. The knowledge graph query module matches the identified and merged question vocabulary with the predefined question intent. After the match is successful, it queries the answer-related content in the knowledge graph established based on Neo4J. If the query is successful, the query result is passed to the answer cache and output module. If the match or query fails, go to path three to generate the answer to the question; Path three of the question-answer matching generation module specifically includes a document fragment query and re-ranking module and a large language model answer generation module: the document fragment query and re-ranking module calls the question embedding vector, matches the content in the main vector database and re-ranks it; the main vector database is established based on pgvector, storing split and cleaned policy document fragments; the matching process uses the cosine similarity formula to calculate the similarity between the question and the policy document fragment; based on the similarity calculation results, the module takes the 30 fragments with the highest similarity and returns them, uses the cross-coding model BGE Reranker to re-rank the 30 results with the highest similarity, and then takes the top 5 re-ranked results and passes them to the large language model answer generation module. In the large language model answer generation module, the 5 policy document fragments with the highest similarity and the preset prompt are submitted to the large language model, and the final answer is generated by the large language model and passed to the answer cache and output module.

10. The intelligent policy question answering system based on retrieval enhancement according to claim 6 is characterized in that: In the answer output and cache module: The answer output and cache module includes an answer output module and a question and answer cache module: when the answer output module receives the answer delivered by path one of the question and answer matching generation module, it directly outputs the answer to the user; when the answer output module receives the answer content delivered by path two of the question and answer matching generation module, it fills the content into a preset answer template and then outputs it to the user, and the question and answer cache module saves the question and answer to the question and answer cache library; when the answer output module receives the answer delivered by path three of the question and answer matching generation module, it outputs the answer to the user and the question and answer cache module saves the question and answer to the question and answer cache library.

Images