Government affair intelligent question answering method and device, storage medium and computer equipment

Abstract

According to the government affair intelligent question and answer method and device, the storage medium and the computer equipment, after the question information input by the target user is obtained, the question type of the question information can be determined firstly, when the question type is the government affair question, the key information of the government affair question can be extracted, knowledge base retrieval is carried out based on the key information, and the user experience is improved. After a retrieval result is obtained, if the retrieval result contains related knowledge fragments but key information is incomplete, gradually deriving answer information according to an inference chain in a preset case library; if the retrieval result contains the related knowledge fragments and the key information is complete, corresponding answer information can be generated according to the retrieval result; if the retrieval result does not contain the related knowledge fragments, the answer is refused, so that misleading information is prevented from being provided for the target user, and the reliability of the government affair service is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a government affairs intelligent question-answering method, apparatus, storage medium and computer equipment. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the rise of large-scale language models (LLMs), the government service sector is facing a revolution in digital transformation. These large models are trained through deep learning algorithms and can understand and generate natural language, providing efficient and accurate information processing capabilities. As one of the applications of this technology, government digital humans are gradually becoming an important tool for improving government service efficiency and citizen experience.

[0003] At present, traditional question-answering systems usually rely on keyword matching or rule-based methods, which cannot deeply understand the user's intentions, resulting in low accuracy of answers. Although the government question-answering system based on Retrieval-Augmented Generation (RAG) technology has improved the response speed and convenience of information acquisition to a certain extent, it still needs to be improved in terms of accuracy and reliability. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies, especially the technical deficiencies of low accuracy and reliability of the government affairs question and answer system in the prior art.

[0005] The present application provides a government affairs intelligent question-answering method, the method comprising:

[0006] Obtaining question information input by a target user and determining the question type of the question information;

[0007] When the type of the question is a government affairs question, extracting key information of the government affairs question, and performing a knowledge base search based on the key information to obtain a search result;

[0008] If the search results contain relevant knowledge fragments but the key information is incomplete, the answer information is gradually derived according to the reasoning chain in the preset case library;

[0009] If the search results contain relevant knowledge fragments and the key information is complete, generating corresponding answer information according to the search results;

[0010] If the search results do not contain relevant knowledge fragments, the answer will be refused.

[0011] Optionally, performing a knowledge base search based on the key information to obtain a search result includes:

[0012] Obtaining historical conversations input by the target user;

[0013] Rewrite the key information based on the historical conversation to obtain the user question;

[0014] Performing vector search and keyword search on the user question respectively to obtain vector search results and keyword search results;

[0015] The vector search result and the keyword search result are fused, and a final search result is determined according to the fusion result.

[0016] Optionally, rewriting the key information based on the conversation history to obtain the user question includes:

[0017] Get pre-configured question rephrasing models;

[0018] The question rewriting model is used to rewrite the key information in multiple dimensions according to the historical conversation to obtain the user question.

[0019] Optionally, performing vector search and keyword search on the user question respectively to obtain vector search results and keyword search results includes:

[0020] Vectorizing the user question, and performing semantic similarity retrieval based on the feature vector to obtain a vector retrieval result;

[0021] Keywords are extracted from the user's question, and a knowledge base search is performed based on the keywords to obtain keyword search results.

[0022] Optionally, the fusing the vector search result and the keyword search result, and determining a final search result according to the fusion result, includes:

[0023] fusing the vector search result and the keyword search result through a pre-configured re-ranking model to obtain a plurality of fusion results;

[0024] The similarity of each fusion result is calculated, and according to the similarity of each fusion result, multiple fusion results with similarity higher than a preset similarity threshold are selected as the final retrieval results.

[0025] Optionally, generating corresponding answer information according to the search result includes:

[0026] Get a pre-configured answer generation model;

[0027] The search result is input into the answer generation model to obtain answer information corresponding to the search result output by the answer generation model.

[0028] Optionally, the method further comprises:

[0029] Calculate the probability distribution corresponding to each answer information;

[0030] Calculate the overall confidence based on the probability distribution corresponding to all the answer information;

[0031] Comparing the overall confidence with a preset confidence threshold, and refusing to answer if the overall confidence is lower than the preset confidence threshold;

[0032] If the overall confidence is not lower than the preset confidence threshold, the key content in each answer information is continuously compared with the relevant information in the knowledge base, and whether to refuse to answer is determined based on the first comparison result.

[0033] Optionally, when there are multiple answer generation models, and answer information corresponding to the search result is outputted through each answer generation model, the method further includes:

[0034] Compare the answer information generated by each answer generation model, and determine the reliability of the answer information generated this time according to the comparison result;

[0035] When the reliability is higher than a preset reliability threshold, one of the answer information is output;

[0036] When the reliability is lower than the preset reliability threshold, the answer is rejected.

[0037] Optionally, the method further comprises:

[0038] In the process of gradually deriving the answer information, monitor whether each step of the reasoning process is reasonable and logically consistent;

[0039] If it is unreasonable and / or logically inconsistent, adjust the reasoning process;

[0040] If it is reasonable or logically consistent, continue with the reasoning.

[0041] Optionally, the method further comprises:

[0042] Acquire context information input by the target user;

[0043] The obtained answer information is compared with the context information, and it is determined whether to refuse to answer based on the second comparison result.

[0044] Optionally, the method further comprises:

[0045] Obtaining feedback results from the target user in response to the answer information;

[0046] The system is updated according to the feedback results.

[0047] Optionally, the method further comprises:

[0048] When the question type is a small talk question, answer information corresponding to the small talk question is generated by a pre-configured large language model.

[0049] The present application also provides a government affairs intelligent question-answering device, comprising:

[0050] An information acquisition module is used to acquire question information input by a target user and determine the question type of the question information;

[0051] A knowledge base search module, for extracting key information of the government affairs question when the question type is a government affairs question, and searching the knowledge base based on the key information to obtain a search result;

[0052] A case library matching module, for gradually deriving answer information according to the reasoning chain in the preset case library if the search result contains relevant knowledge fragments but the key information is incomplete;

[0053] An answer generation module, configured to generate corresponding answer information according to the search results if the search results contain relevant knowledge fragments and the key information is complete;

[0054] The answer refusal module is used to refuse to answer if the search result does not contain relevant knowledge fragments.

[0055] The present application also provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the government affairs intelligent question and answer method as described in any one of the above embodiments.

[0056] The present application also provides a computer device, comprising: one or more processors, and a memory;

[0057] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the government affairs intelligent question and answer method as described in any one of the above embodiments are performed.

[0058] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0059] The government affairs intelligent question-and-answer method, apparatus, storage medium, and computer equipment provided by the present application can first determine the question type of the question information after obtaining the question information input by the target user. When the question type is a government affairs question, the key information of the government affairs question can be extracted, and a knowledge base search can be performed based on the key information. After obtaining the search results, if the search results contain relevant knowledge fragments but the key information is incomplete, the answer information can be gradually derived according to the reasoning chain in the preset case library; if the search results contain relevant knowledge fragments and the key information is complete, the corresponding answer information can be generated according to the search results; if the search results do not contain relevant knowledge fragments, the answer will be refused to avoid providing misleading information to the target user, thereby improving the reliability of government services. During this process, this application can not only provide the system with a broad and stable source of knowledge through the knowledge base to ensure the authority and consistency of the answers, but also use the actual cases and reasoning chains in the case library to ensure that the model can refer to the ideas of similar cases when answering, and gradually derive accurate answers, thereby effectively responding to complex government affairs scenarios, reducing the problem of inaccurate answers caused by insufficient information, and enabling the system to flexibly adapt to the personalized needs of users, thereby generating more targeted answers in government affairs scenarios, thereby enhancing satisfaction and trust in government services, and providing technical support for the intelligent transformation of government affairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0061] Figure 1 A flowchart of an intelligent government question-answering method provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of a process for performing knowledge base retrieval based on key information provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the entire process of the government affairs intelligent question-answering method provided in the embodiment of the present application;

[0064] Figure 4 A schematic diagram of the structure of an intelligent government question-answering device provided in an embodiment of the present application;

[0065] Figure 5 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] In one embodiment, Figure 1 As shown, Figure 1 A flowchart of a government affairs intelligent question-answering method provided in an embodiment of the present application; the present application provides a government affairs intelligent question-answering method, which may include:

[0068] S110: Obtain question information input by the target user, and determine the question type of the question information.

[0069] In this step, the government affairs question and answer system carrying the government affairs intelligent question and answer method of this application can be applied to a digital human all-in-one machine, a web page or a mini-program to provide users with automated consulting services, such as government convenience service consultation, entry and exit business consultation, etc.

[0070] Therefore, when the target user logs in to the government affairs question and answer system and enters the question information, this application can first determine the question type of the question information, and then execute different business processes according to different types of questions. For example, this application can divide the question types into government affairs questions and chat questions. For government affairs questions, intelligent question and answer are performed through the corresponding business process, and for chat questions, intelligent question and answer are performed through the corresponding business process. This can not only flexibly adapt to the personalized needs of users, but also generate more targeted answers in government affairs scenarios.

[0071] S120: When the question type is a government affairs question, extract key information of the government affairs question, and perform a knowledge base search based on the key information to obtain a search result.

[0072] In this step, after obtaining the question information input by the target user through S110 and determining the question type of the question information, the present application can extract the key information of the government affairs question when the question type of the question information is a government affairs question, and perform a knowledge base search based on the key information to obtain the search results.

[0073] Specifically, this application can first extract the key information of the government affairs questions asked by the target users. The extraction process can be carried out through a large model. The extracted content includes user core information, such as extracting keywords, identifying question types (such as information query, policy consultation, processing procedures, etc.), analyzing user background identity (nationality, age, household registration, etc.), etc., and then perform knowledge base search based on the extracted key information to obtain search results.

[0074] The knowledge base here is mainly used to store common knowledge information such as general government policies, laws and regulations, service guidelines, etc. The knowledge base can provide a broad and stable source of knowledge for the system to ensure the authority and consistency of the answers.

[0075] Since the knowledge base is suitable for static queries, the system can find relevant answers through semantic vectors and question keywords. When the user's questions are more direct or clear in content, the knowledge base can provide quick and accurate answers, such as "What are the basic materials for applying for a passport?", "What are the conditions for applying for a residence permit?", etc. For such questions, the knowledge base can quickly provide accurate answers.

[0076] Here is an example:

[0077] Title: Passport application materials

[0078] content:

[0079] 1. Adults: ID card, household registration book, photos that meet the specifications, passport application form

[0080] 2. Minors: ID card or household registration book, guardian's ID card, photos that meet the requirements, passport application form

[0081] S130: If the search results contain relevant knowledge fragments but the key information is incomplete, the answer information is gradually derived according to the reasoning chain in the preset case library.

[0082] In this step, key information of government affairs issues is extracted through S120, and a knowledge base search is performed based on the key information. After the search results are obtained, since the search results include multiple possibilities, such as containing relevant knowledge fragments or not containing relevant knowledge fragments, the present application can perform different business processes for different search results.

[0083] For example, if the search results contain relevant knowledge fragments but the key information is incomplete, this application can gradually derive the answer information according to the reasoning chain in the preset case library. Among them, the case library of this application focuses on storing specific consulting scenarios and response strategies, aiming to simulate the human thinking process through Chain-of-Thought (CoT) multi-step reasoning technology, and enhance the model's ability to deal with complex problems, especially in situations where user information is incomplete and personalized needs are obvious. The case library can provide the model with solutions and logical chains for similar problems, helping the system to generate explanatory answers. And each case in the case library contains a scenario description, user background, and reasoning steps. Therefore, the case library is more scenario-based and logical than the knowledge base, so as to achieve multi-step reasoning based on specific scenarios.

[0084] Examples of case libraries for this application are as follows:

[0085] Case ID: PASSPORT_APP_001

[0086] Scenario description: User asks how to apply for a passport

[0087] User background: Minor, Guangzhou resident, 14 years old

[0088] Reasoning steps (CoT steps):

[0089] 1. Confirm age: Prompt to confirm whether the user is under 18 years old.

[0090] 2. Prompt guardian information: If the child is a minor, the guardian’s identity certificate is required.

[0091] 3. Confirm the preparation of materials: whether the user holds an ID card or household registration booklet, and whether he has photos and application forms that meet the specifications.

[0092] Provide the processing steps: provide the processing location and the required appointment or on-site processing instructions.

[0093] In another embodiment of the present application, when consulting on passport application, the system can first determine whether the user's age meets the application conditions, then confirm the user's household registration, and then gradually prompt the required materials and specific procedures. If it is found during the reasoning process that the new information provided by the user affects the current reasoning step (for example, the user adds special circumstances such as expedited requirements), the system can dynamically adjust the reasoning path to ensure the personalization and accuracy of the answer.

[0094] Among them, the above-mentioned CoT (Chain-of-Thought) is a multi-step reasoning technology that decomposes complex problems into a series of logical steps by gradually guiding its thinking process in a large model, thereby improving the accuracy and explainability of the answer. In the question-answering process, the CoT method can not only output the final answer, but also provide a reasoning chain for generating answers, so that the model has higher accuracy in understanding complex problems. Especially in government affairs scenarios, the reasoning chain of CoT helps the model to clearly judge the information and conditions required for each step, thereby reducing errors caused by incomplete or ambiguous information.

[0095] It is understandable that since the consultation and handling of many government services require users to provide relatively detailed personal information (such as identity, age, household registration information, etc.), when users cannot clearly provide all the necessary information during the question-and-answer process, the answers generated by the system will be incomplete or inaccurate, which will affect the user experience. In addition, in complex government scenarios, the coverage of the knowledge base is often limited and cannot include answers to all government affairs. This incomplete knowledge base can easily lead to the system being unable to provide accurate responses, especially for some personalized policy and regulatory issues, thereby reducing the effectiveness of the question-and-answer session.

[0096] In this case, the present application, through the actual cases in the case library and the multi-step reasoning chain of CoT, can not only ensure that the model can refer to the ideas of similar cases when answering, and gradually derive accurate answers, so as to effectively deal with complex government affairs scenarios, but also reduce the problem of inaccurate answers caused by insufficient information. When the information input by the user is insufficient, through the linkage of the case library and CoT technology, the user is prompted to provide key information to complete the reasoning process, so that the system can flexibly adapt to the personalized needs of the user, thereby generating more targeted answers in government affairs scenarios. In addition, this application improves the user experience in the government affairs question and answer process by improving the accuracy, reliability and interpretability of the government affairs question and answer system, thereby effectively enhancing the satisfaction and trust of government services, and providing technical support for the intelligent transformation of government affairs.

[0097] S140: If the search results contain relevant knowledge fragments and the key information is complete, corresponding answer information is generated according to the search results.

[0098] In this step, the key information of the government affairs issue is extracted through S120, and the knowledge base is searched based on the key information. After the search results are obtained, if the search results contain relevant knowledge fragments and the key information is complete, the present application can generate corresponding answer information based on the search results. Of course, in the process of generating answer information, in order to reduce the possibility of generating untrue or even misleading answers, the present application can further perform logic and fact verification on the generated answers to ensure that there are no hallucinations, that is, to avoid generating misleading or false answers.

[0099] S150: If the search results do not contain relevant knowledge fragments, refuse to answer.

[0100] In this step, key information of government affairs issues is extracted through S120, and a knowledge base search is performed based on the key information. After obtaining the search results, if the search results do not contain relevant knowledge fragments, the answer is refused to avoid providing misleading information to the target users, thereby improving the reliability of government services.

[0101] Furthermore, the present application can also clarify questions or supplement the knowledge base / case base when the search results do not contain relevant knowledge fragments, so as to improve the types of cases in the knowledge base and case base, thereby improving the accuracy and reliability of subsequent searches.

[0102] In the above embodiment, after obtaining the question information input by the target user, the question type of the question information can be determined first. When the question type is a government question, the key information of the government question can be extracted, and the knowledge base can be searched based on the key information. After obtaining the search results, if the search results contain relevant knowledge fragments but the key information is incomplete, the answer information can be gradually derived according to the reasoning chain in the preset case library; if the search results contain relevant knowledge fragments and the key information is complete, the corresponding answer information can be generated according to the search results; if the search results do not contain relevant knowledge fragments, the answer is rejected to avoid providing misleading information to the target user, thereby improving the reliability of government services. In this process, the present application can not only provide the system with a wide and stable source of knowledge through the knowledge base to ensure the authority and consistency of the answer, but also ensure that the model can refer to the ideas of similar cases when answering through the actual cases and reasoning chains in the case library, and gradually derive accurate answers, thereby effectively coping with complex government scenarios, reducing the problem of inaccurate answers caused by insufficient information, and enabling the system to flexibly adapt to the personalized needs of users, thereby generating more targeted answers in government scenarios, thereby enhancing the satisfaction and trust of government services, and providing technical support for the intelligent transformation of government affairs.

[0103] In one embodiment, Figure 2 As shown, Figure 2 A schematic diagram of a process of performing knowledge base search based on key information provided in an embodiment of the present application; performing knowledge base search based on the key information in S120 to obtain search results may include:

[0104] S121: Obtain the historical dialogue input by the target user.

[0105] S122: Rewrite the key information based on the historical conversation to obtain the user question.

[0106] S123: Perform vector search and keyword search on the user question respectively to obtain vector search results and keyword search results.

[0107] S124: Fusing the vector search result and the keyword search result, and determining the final search result according to the fusion result.

[0108] In this embodiment, when searching the knowledge base based on key information, the historical dialogue input by the target user can be obtained first, and then the key information can be rewritten based on the historical dialogue. After obtaining the user's question, since the knowledge base is suitable for static queries, the present application can find relevant answers through vector retrieval and keyword retrieval. After obtaining the vector retrieval results and keyword retrieval results, the present application can merge the two retrieval results to determine the final retrieval results based on the fusion results.

[0109] Specifically, since the question information input by the target user is mostly colloquial expressions, in order to rewrite the user's colloquial expressions into standard business questions, in order to improve the accuracy of knowledge base and case library retrieval. The present application can rewrite key information based on historical conversations, and perform vector retrieval and keyword retrieval based on the rewritten user questions. It can be understood that vector retrieval and keyword retrieval are two different retrieval methods. Vector retrieval is at the semantic level, and even if the user's description method is different, it can identify semantically related content; while keyword retrieval is based on traditional text matching and is good at precise matching. Therefore, after the present application integrates two different retrieval methods, the accuracy of the retrieval can be effectively improved.

[0110] In one embodiment, in S122, the key information is rewritten based on the conversation history to obtain the user question, which may include:

[0111] S1221: Obtain a pre-configured question rewriting model.

[0112] S1222: Utilize the question rewriting model to rewrite the key information in multiple dimensions according to the historical conversation to obtain a user question.

[0113] In this embodiment, in order to rewrite the user's colloquial expression into a standard business question so as to improve the accuracy of the knowledge base and case base retrieval, the application can rewrite the key information based on the historical dialogue.

[0114] Specifically, the present application can obtain a pre-configured question rewriting model, which is obtained after training with a large number of historical conversations and key information. Therefore, after calling the question rewriting model, the present application can use the question rewriting model to perform multi-dimensional question rewriting on key information based on the historical conversations of the target user, and then obtain the final user question.

[0115] For example, this application can rewrite the user's current question based on historical conversations, including text correction, reference disambiguation, information completion, etc. For example, if a user asks: What should I do if I lose my passport? Combined with the context, it can be rewritten as: The process of handling lost passports for foreigners. This is more conducive to improving the accuracy of subsequent retrieval.

[0116] In one embodiment, performing vector search and keyword search on the user question in S123 respectively to obtain vector search results and keyword search results may include:

[0117] S1231: Feature vectorization is performed on the user question, and semantic similarity retrieval is performed based on the feature vector to obtain a vector retrieval result.

[0118] S1232: extract keywords from the user question, and perform a knowledge base search based on the keywords to obtain keyword search results.

[0119] In this embodiment, in the process of performing vector retrieval and keyword retrieval on user questions respectively, the user questions can be feature vectorized, and semantic similarity retrieval can be performed based on the feature vectors to obtain vector retrieval results. Keywords can also be extracted from the user questions, and knowledge base retrieval can be performed based on the keywords to obtain keyword retrieval results.

[0120] Furthermore, when performing feature vectorization on user questions, a specially trained vectorization model can be used to vectorize the user questions. The knowledge base data can also use the same model for vectorization and indexing. By vectorizing user questions, massive knowledge base data can be retrieved quickly and accurately.

[0121] In one embodiment, in S124, the vector search result and the keyword search result are merged, and the final search result is determined according to the merged result, which may include:

[0122] S1241: Fusing the vector search result and the keyword search result through a pre-configured re-ranking model to obtain a plurality of fusion results.

[0123] S1242: Calculate the similarity of each fusion result, and select multiple fusion results with similarities higher than a preset similarity threshold as final retrieval results according to the similarities of the various fusion results.

[0124] In this embodiment, when the vector search results and the keyword search results are fused, they can be fused through a pre-configured re-ranking model, and the similarity scores of the fused results are calculated, and the N results with the highest scores are selected as the final search results.

[0125] It is understandable that the re-ranking model is a model that is widely used in many fields such as information retrieval and recommendation systems. Its main function is to re-rank the results that have been initially screened to optimize the display order of the results so that the results that are more in line with user expectations and needs are ranked first. For example, in the government question-and-answer system of this application, some different knowledge fragments and cases can be retrieved through vector retrieval results and keyword retrieval results. At this time, the re-ranking model can re-rank these knowledge fragments according to various factors such as the quality, relevance, and user preferences of each knowledge fragment and case, and rank the knowledge fragments that are most likely to meet user needs higher.

[0126] Next, the present application can perform a similarity score on each fusion result to obtain a similarity score for each fusion result. The present application can select the N results with the highest scores as the final search results to improve the search accuracy and reliability.

[0127] In one embodiment, generating corresponding answer information according to the search result in S140 may include:

[0128] S141: Obtain a pre-configured answer generation model.

[0129] S142: Input the search result into the answer generation model to obtain answer information corresponding to the search result output by the answer generation model.

[0130] In this embodiment, when generating corresponding answer information based on the retrieval results, a pre-configured answer generation model can be obtained, and the retrieval results can be input into the answer generation model so that answer information corresponding to the retrieval results can be quickly generated through the answer generation model.

[0131] Among them, the answer generation model here can be based on the sequence-to-sequence (Seq2Seq) architecture, which includes an encoder and a decoder. The encoder converts the input retrieval results into a fixed-length vector representation. This vector contains the semantic information of the input text, and the decoder generates answers word by word or character by character based on this vector. It can also be a pre-trained language model (Transformer-based). Models such as ERT (BidirectionalEncoder Representations from Transformers) and GPT (Generative PretrainedTransformer) series are all based on the Transformer architecture. These models are pre-trained on large-scale text data to learn the grammar, semantics and knowledge of the language. In the answer generation task, they can be fine-tuned to adapt to specific tasks. For example, the GPT model is a generative model that can handle long sequence texts well through the self-attention mechanism. When generating answers, it will generate reasonable answers based on the input prompts and the language knowledge and patterns learned in the pre-training stage. BERT is mainly used to understand the semantics of text. By bidirectionally encoding the input text, it provides rich semantic information for answer generation. It is very useful in some scenarios where understanding the contextual semantics is required to generate answers. Therefore, this application can select a specific model to generate answer information according to actual conditions, and there is no restriction here.

[0132] In one embodiment, the method may further include:

[0133] S143: Calculate the probability distribution corresponding to each answer information.

[0134] S144: Calculate the overall confidence level according to the probability distribution corresponding to all the answer information.

[0135] S145: Compare the overall confidence with a preset confidence threshold, and if the overall confidence is lower than the preset confidence threshold, refuse to answer.

[0136] S146: If the overall confidence is not lower than the preset confidence threshold, then continue to compare the key content in each answer information with the relevant information in the knowledge base, and determine whether to refuse to answer based on the first comparison result.

[0137] In this embodiment, after obtaining multiple answer information corresponding to the search results through the answer generation model, the present application can also calculate the probability distribution of each answer information, and then calculate the overall confidence according to the probability distribution corresponding to all the answer information, and then compare the overall confidence with the preset confidence threshold. If the overall confidence is lower than the preset confidence threshold, the answer is rejected to improve the reliability of the system; if the overall confidence is not lower than the preset confidence threshold, the key content in each answer information is continued to be compared with the relevant information in the knowledge base, and whether to reject the answer is determined based on the first comparison result. For example, if the answer information contains specific conditions, materials or numbers, but the referenced knowledge fragment does not, the answer is considered to be incorrect, which can further improve the accuracy of the answer result.

[0138] Among them, when this application calculates the overall confidence based on the probability distribution corresponding to all the answer information, the specific formula is as follows:

[0139]

[0140] In the above formula, is the overall confidence, For the Answer information, P For the The probability distribution corresponding to the answer information.

[0141] In one embodiment, when there are multiple answer generation models, and answer information corresponding to the search result is outputted through each answer generation model, the method may further include:

[0142] S147: Compare the answer information generated by each answer generation model, and determine the reliability of the answer information generated this time based on the comparison result.

[0143] S148: When the reliability is higher than a preset reliability threshold, one of the answer information is output.

[0144] S149: When the reliability is lower than the preset reliability threshold, the answer is rejected.

[0145] In this embodiment, when generating answer information, answers can also be generated through multiple models and compared. If the output consistency of multiple models is high, the reliability of the answer is increased. Otherwise, the credibility of the generated content is considered to be low.

[0146] Specifically, the present application can compare the answer information generated by each answer generation model, and determine the reliability of the answer information generated this time based on the comparison result. When the reliability is higher than the preset reliability threshold, one of the answer information is output; when the reliability is lower than the preset reliability threshold, the answer is refused, thereby increasing the reliability of the present system.

[0147] In one embodiment, the method may further include:

[0148] S150: In the process of gradually deriving answer information, monitor whether each step of the reasoning process is reasonable and logically consistent.

[0149] S151: If it is unreasonable and / or logically inconsistent, adjust the reasoning process.

[0150] S152: If it is reasonable or logically consistent, continue reasoning.

[0151] In this embodiment, when using CoT for reasoning, the present application can also monitor whether each step in the reasoning process is reasonable and consistent. If the reasoning logic of a certain step is inconsistent with the previous step, hallucinations may occur, and the system should identify and adjust.

[0152] Understandably, existing large language models may experience so-called "hallucination" when generating answers, that is, the model generates untrue or even misleading answers in the absence of explicit knowledge or sufficient contextual information. For government question-and-answer systems, this "hallucination" problem is particularly serious, because incorrect answers may mislead users, leading to user behavior deviations, affecting the credibility of government services and user experience.

[0153] Therefore, in the process of gradually deriving the answer information, this application can monitor whether each step of the reasoning process is reasonable and logically consistent. This process can be implemented by cross-comparison using multiple models, or by other methods. The specific method can be selected according to the actual situation and is not limited here. When it is determined that there is unreasonableness and / or logical inconsistency, the reasoning process can be adjusted to make it logically reasonable. If it is reasonable and logically consistent, the reasoning can continue, thereby effectively improving the credibility and user experience of government services.

[0154] In one embodiment, the method may further include:

[0155] S160: Acquire context information input by the target user.

[0156] S161: Compare the obtained answer information with the context information, and determine whether to refuse to answer based on the second comparison result.

[0157] In this embodiment, in order to avoid the so-called "hallucination" phenomenon that may occur when the existing large language model generates answers, the present application can also obtain the context information input by the target user, and compare the obtained answer information with the context information, and then determine whether to refuse to answer based on the second comparison result.

[0158] For example, this application compares the context information with the generated answer. If it finds that the answer is obviously inconsistent with the context (such as time, place, identity, etc.), it can be determined as a possible hallucination. At this time, the answer can be refused, thereby effectively improving the credibility of government services.

[0159] In one embodiment, the method may further include:

[0160] S170: Obtaining the feedback result made by the target user in response to the answer information.

[0161] S171: Update the system according to the feedback result.

[0162] In this embodiment, after answering the target user's questions, feedback results on the target user's satisfaction and accuracy of the answer information can also be obtained. The user's feedback results can be used as the basis for model training and updating, so as to further improve the accuracy and reliability of the government question and answer system.

[0163] In one embodiment, Figure 3 As shown, Figure 3 A schematic diagram of the entire process of the government affairs intelligent question-answering method provided in an embodiment of the present application; the method may also include:

[0164] S180: When the question type is a small talk question, answer information corresponding to the small talk question is generated through a pre-configured large language model.

[0165] In this embodiment, when the question type is a small talk question, answer information corresponding to the small talk question can be generated by a pre-configured large language model.

[0166] It is understandable that the large language model used by the system can use closed source models, such as ChatGPT and Wenxin Yiyan, or open source models, such as Qwen2.5, llama3.1, etc., and fine-tune through business data to further improve the accuracy of each link.

[0167] The government affairs intelligent question and answer device provided in an embodiment of the present application is described below. The government affairs intelligent question and answer device described below and the government affairs intelligent question and answer method described above can be referenced to each other.

[0168] In one embodiment, Figure 4 As shown, Figure 4A schematic diagram of the structure of an intelligent government question-answering device provided in an embodiment of the present application; the present application also provides an intelligent government question-answering device, which may include an information acquisition module 210, a knowledge base retrieval module 220, a case base matching module 230, an answer generation module 240, and a refusal to answer module 250, specifically including the following:

[0169] The information acquisition module 210 is used to acquire question information input by the target user and determine the question type of the question information.

[0170] The knowledge base search module 220 is used to extract key information of the government affairs question when the question type is a government affairs question, and perform a knowledge base search based on the key information to obtain a search result.

[0171] The case base matching module 230 is used to gradually derive answer information according to the reasoning chain in the preset case base if the search result contains relevant knowledge fragments but the key information is incomplete.

[0172] The answer generation module 240 is used to generate corresponding answer information according to the search results if the search results contain relevant knowledge fragments and the key information is complete.

[0173] The answer rejection module 250 is used to reject the answer if the search result does not contain the relevant knowledge fragment.

[0174] In the above embodiment, after obtaining the question information input by the target user, the question type of the question information can be determined first. When the question type is a government question, the key information of the government question can be extracted, and the knowledge base can be searched based on the key information. After obtaining the search results, if the search results contain relevant knowledge fragments but the key information is incomplete, the answer information can be gradually derived according to the reasoning chain in the preset case library; if the search results contain relevant knowledge fragments and the key information is complete, the corresponding answer information can be generated according to the search results; if the search results do not contain relevant knowledge fragments, the answer is rejected to avoid providing misleading information to the target user, thereby improving the reliability of government services. In this process, the present application can not only provide the system with a wide and stable source of knowledge through the knowledge base to ensure the authority and consistency of the answer, but also ensure that the model can refer to the ideas of similar cases when answering through the actual cases and reasoning chains in the case library, and gradually derive accurate answers, thereby effectively coping with complex government scenarios, reducing the problem of inaccurate answers caused by insufficient information, and enabling the system to flexibly adapt to the personalized needs of users, thereby generating more targeted answers in government scenarios, thereby enhancing the satisfaction and trust of government services, and providing technical support for the intelligent transformation of government affairs.

[0175] In one embodiment, the present application also provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the government affairs intelligent question and answer method as described in any one of the above embodiments.

[0176] In one embodiment, the present application also provides a computer device, including: one or more processors, and a memory.

[0177] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the government affairs intelligent question and answer method as described in any one of the above embodiments are performed.

[0178] Indicatively, Figure 5 As shown, Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 may be provided as a server. Figure 5 The computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by a memory 301 for storing instructions executable by the processing component 302, such as an application. The application stored in the memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the government affairs intelligent question answering method of any of the above embodiments.

[0179] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0180] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0181] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0182] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0183] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A government affairs intelligent question-answering method, characterized in that: The method comprises: Obtaining question information input by a target user and determining the question type of the question information; When the type of the question is a government affairs question, extracting key information of the government affairs question, and performing a knowledge base search based on the key information to obtain a search result; If the search results contain relevant knowledge fragments but the key information is incomplete, the answer information is gradually derived according to the reasoning chain in the preset case library; If the search results contain relevant knowledge fragments and the key information is complete, generating corresponding answer information according to the search results; If the search results do not contain relevant knowledge fragments, the answer will be refused.

2. The government affairs intelligent question-answering method according to claim 1 is characterized in that: The step of performing a knowledge base search based on the key information to obtain a search result includes: Obtaining historical conversations input by the target user; Rewrite the key information based on the historical conversation to obtain the user question; Performing vector search and keyword search on the user question respectively to obtain vector search results and keyword search results; The vector search result and the keyword search result are fused, and a final search result is determined according to the fusion result.

3. The government affairs intelligent question-answering method according to claim 2 is characterized in that: The step of rewriting the key information based on the conversation history to obtain the user question includes: Get pre-configured question rephrasing models; The question rewriting model is used to rewrite the key information in multiple dimensions according to the historical conversation to obtain the user question.

4. The government affairs intelligent question-answering method according to claim 2, characterized in that: The performing vector search and keyword search on the user question respectively to obtain vector search results and keyword search results includes: Vectorizing the user question, and performing semantic similarity retrieval based on the feature vector to obtain a vector retrieval result; Keywords are extracted from the user's question, and a knowledge base search is performed based on the keywords to obtain keyword search results.

5. The government affairs intelligent question-answering method according to claim 2 is characterized in that: The step of fusing the vector search result and the keyword search result, and determining a final search result according to the fusion result, includes: fusing the vector search result and the keyword search result through a pre-configured re-ranking model to obtain a plurality of fusion results; The similarity of each fusion result is calculated, and according to the similarity of each fusion result, multiple fusion results with similarity higher than a preset similarity threshold are selected as the final retrieval results.

6. The government affairs intelligent question-answering method according to claim 1, characterized in that: The generating corresponding answer information according to the search result includes: Get a pre-configured answer generation model; The search result is input into the answer generation model to obtain answer information corresponding to the search result output by the answer generation model.

7. The government affairs intelligent question-answering method according to claim 6, characterized in that: The method further comprises: Calculate the probability distribution corresponding to each answer information; Calculate the overall confidence based on the probability distribution corresponding to all the answer information; Comparing the overall confidence with a preset confidence threshold, and refusing to answer if the overall confidence is lower than the preset confidence threshold; If the overall confidence is not lower than the preset confidence threshold, the key content in each answer information is continuously compared with the relevant information in the knowledge base, and whether to refuse to answer is determined based on the first comparison result.

8. The government affairs intelligent question-answering method according to claim 6, characterized in that: When there are multiple answer generation models, and answer information corresponding to the search result is outputted respectively by each answer generation model, the method further includes: Compare the answer information generated by each answer generation model, and determine the reliability of the answer information generated this time according to the comparison result; When the reliability is higher than a preset reliability threshold, one of the answer information is output; When the reliability is lower than the preset reliability threshold, the answer is rejected.

9. The government affairs intelligent question-answering method according to claim 1, characterized in that: The method further comprises: In the process of gradually deriving the answer information, monitor whether each step of the reasoning process is reasonable and logically consistent; If it is unreasonable and / or logically inconsistent, adjust the reasoning process; If it is reasonable or logically consistent, continue with the reasoning.

10. The government affairs intelligent question-answering method according to any one of claims 1 to 9, characterized in that: The method further comprises: Acquire context information input by the target user; The obtained answer information is compared with the context information, and it is determined whether to refuse to answer based on the second comparison result.

11. The government affairs intelligent question-answering method according to any one of claims 1 to 9, characterized in that: The method further comprises: Obtaining feedback results from the target user in response to the answer information; The system is updated according to the feedback results.

12. The government affairs intelligent question-answering method according to any one of claims 1 to 9, characterized in that: The method further comprises: When the question type is a small talk question, answer information corresponding to the small talk question is generated by a pre-configured large language model.

13. An intelligent government question-answering device, characterized in that: include: An information acquisition module is used to acquire question information input by a target user and determine the question type of the question information; A knowledge base search module, for extracting key information of the government affairs question when the question type is a government affairs question, and searching the knowledge base based on the key information to obtain a search result; A case library matching module, for gradually deriving answer information according to the reasoning chain in the preset case library if the search result contains relevant knowledge fragments but the key information is incomplete; An answer generation module, configured to generate corresponding answer information according to the search results if the search results contain relevant knowledge fragments and the key information is complete; The answer refusal module is used to refuse to answer if the search result does not contain relevant knowledge fragments.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the government affairs intelligent question-answering method as described in any one of claims 1 to 12.

15. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the government affairs intelligent question-answering method as described in any one of claims 1 to 12 are performed.

Images