Government affair question answering system based on deep government affair thinking

Abstract

The invention provides a government affair question-answering system based on deep government affair thinking, and the system comprises a data construction module which is used for generating a high-quality government affair question-answering data set from multi-source data through data preprocessing, screening and classification; the knowledge RAG preprocessing module is used for performing washing, segmentation and index construction on the work order text data and the knowledge document data so as to support subsequent knowledge retrieval and answer generation; the deep government affair thinking training module endows the large model with the ability of deep government affair thinking by enhancing fine tuning and reinforcement learning of deep government affair thinking, and completes government affair question answering; the reasoning module is used for receiving input of dialogue data and work order data, extracting key information, comprehensively analyzing the information through a deep thinking process, triggering knowledge retrieval, searching policy documents and similar instances in a retrieval knowledge base, and inputting retrieved texts and current contexts into a model to generate answers; the government affair question-answering efficiency and specialty and the citizen satisfaction are improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence and government affairs services, and particularly relates to a government affair question and answer system based on deep government affair thinking. BACKGROUND

[0002] With the promotion of city governance refinement, hotlines play an increasingly important role in gathering citizen demands and coordinating inter-departmental linkage. How to efficiently locate key content between complex regulations and historical cases, and provide feedback to callers in a coherent and credible manner has become a core challenge in the process of integrating smart government and intelligent question and answer technology.

[0003] Under this background, more and more researchers and technical teams have begun to try to use large-scale pre-trained language models to process problem analysis and answer generation in the government hotline scenario. Large models have shown high plasticity in natural language understanding and multi-turn dialogue generation, and combined with the retrieval augmented generation (RAG) technology, they can dynamically access structured or semi-structured knowledge bases to improve the accuracy and authority of the answers. However, in complex government scenarios, simple model combination is still difficult to capture multi-dimensional information, leading to biased answers. However, in practical applications, simply combining large models with RAG often fails to fully capture the multi-dimensional information contained in complex government scenarios: for example, work orders may involve multi-department cross responsibilities, fine-grained regulations, and attribute missing or confusion caused by unclear user expression. In this case, if only relying on dialogue text and conventional retrieval matching, the model may not accurately understand the user's real demands, or may have biased answers due to the failure to dig out potential key points.

[0004] In real-world scenarios, hotline personnel usually "think" about the caller's background, statement focus and potential needs before making diversified processing judgments based on existing policies or case experience when facing citizen inquiries. However, traditional large model reasoning mode does not have an explicit "thinking" stage before answering, and its dialogue generation process is mostly completed in an implicit vector space, making it difficult to examine user clues such as gender, occupation or tone as artificially as a human operator, and unable to actively identify and correct missing or noisy fields.

[0005] Among existing patent technologies, the invention patent with application number CN202411931491.1 provides an answer text generation method, device, equipment, storage medium, program product, and government question-and-answer system. It focuses on answer generation in general fields and implements question-and-answer through a large language model combined with basic text library retrieval. It does not optimize for the particularities of government scenarios (such as overlapping responsibilities of multiple departments, fine-grained matching of legal provisions, and ambiguity of user demands). Moreover, it mainly solves the binary judgment of "whether to search" and does not delve into complex needs such as attribute missing completion, cross-departmental collaborative logical reasoning, and authoritative citation of policies and regulations that are common in government scenarios.

[0006] Patent application number CN202411536129.4 provides a method and system for implementing government Q&A based on a large model and multi-level convergence. This method gradually narrows the scope through three levels of convergence (topic → business → content), relying on a fixed menu list (such as the first-level convergence "food / drug business") and user interaction. Furthermore, during the convergence phase, the large model is only used to match preset menus, without the introduction of dynamic reasoning or information completion. For example, it cannot handle situations where work order attributes are missing or user requests are ambiguous.

[0007] In order to solve the problems existing in the above-mentioned existing technologies, it is necessary to transform the reasoning mechanism of the large model and enhance its self-inference and information completion capabilities before answering, so as to improve the efficiency and professionalism of government Q&A and increase citizen satisfaction. Summary of the Invention

[0008] The present invention aims to overcome the deficiencies of the prior art and provide a government affairs question-answering system based on in-depth government affairs thinking.

[0009] The present invention is achieved in that:

[0010] A government affairs question-answering system based on deep government affairs thinking, including:

[0011] The data construction module is used to generate high-quality government Q&A datasets from multi-source data through data preprocessing, screening, and classification;

[0012] The knowledge RAG preprocessing module is used to clean, segment, and index work order text data and knowledge document data to support subsequent knowledge retrieval and answer generation;

[0013] The deep government thinking training module empowers large models with deep government thinking capabilities through fine-tuning and reinforcement learning, enabling them to answer government questions.

[0014] The reasoning module is used to receive inputs of conversation data and work order data, extract key information, comprehensively analyze this information through a deep thinking process, trigger knowledge retrieval, search for policy documents and similar instances in the retrieval knowledge base, and then input the retrieved text together with the current context into the model to generate answers.

[0015] Furthermore, the data construction module includes:

[0016] The data element analysis unit divides data into three categories: knowledge documents, conversation recordings, and work order text data, and achieves high-quality training and testing of the model through multi-dimensional association;

[0017] The dataset construction unit comprehensively utilizes conversation recording data, work order text data, and knowledge document data, and generates high-quality datasets through multiple screening and preprocessing.

[0018] Furthermore, the knowledge document data includes knowledge <document number>, <title>, <keywords>, <department> and corresponding <questions> and <answers>; the conversation recording data is pre-processed by the ASR model and used to construct the conversation embedding of the instruction prompt template; the work order text data contains attribute element information such as <title>, <content>, <label>, <processing department> and <reply>, which is mainly used to construct the work order attribute embedding of the instruction context prompt template.

[0019] Furthermore, the data set construction unit specifically includes:

[0020] The conversation recording data processing sub-unit, after pre-processing with the ASR model, excludes non-Mandarin, blank, or low-information text, as well as recordings related to "return visits" topics. It then extracts the core "appeal-response" elements of the text and converts and reorganizes it according to Alpaca's instructions and fine-tunes the format.

[0021] The work order text data processing sub-unit uses a multi-round self-feedback classification network to combine and judge the title and content, identify labels such as department and business type, delete common template expressions, and eliminate duplicate and blank information;

[0022] The knowledge document data processing sub-unit covers various types such as notices, business guidelines and service guides. It adopts a deduplication algorithm based on text similarity matching, performs coarse-grained segmentation on the text, and finally saves it in CSV format.

[0023] Furthermore, the knowledge RAG preprocessing module includes a knowledge RAG preprocessing unit that cleans, segments, and constructs indexes for work order text data and knowledge document data;

[0024] The knowledge RAG preprocessing unit specifically includes:

[0025] The text segmentation sub-unit segments the text content according to knowledge items to control the length and semantic integrity of the text blocks;

[0026] Lexicalization subunit: Each sub-segment is lexicalized in a fine-grained manner using the Jieba word segmentation tool and keyword extraction. A vocabulary is defined and mapped to the text position of the corresponding sub-segment.

[0027] The index construction sub-unit uses the inverted index scheme of the Elasticsearch search engine to reversely map the keywords of each sub-segment to the paragraph and specific location to form an index library;

[0028] The vectorization processing subunit optionally introduces vectorization processing to the segmented text, generates word vectors or sentence vectors, and stores them in the vector index.

[0029] Furthermore, the in-depth government affairs thinking training module includes:

[0030] Enhance the fine-tuning unit of deep government affairs thinking, fine-tune the large model base through work order data with complete attributes, and convert work order information into intermediate results of thinking;

[0031] The reinforcement learning unit that enhances deep government thinking uses reinforcement learning methods to enhance the deep thinking ability of large models when processing work orders with incomplete attributes.

[0032] Furthermore, the fine-tuning unit for enhancing in-depth government affairs thinking specifically includes:

[0033] The context prompt instruction template constructs sub-units. The large model uses conversation data to search the knowledge base and build a unified context prompt instruction template. The conversation text, structured attributes, and external knowledge documents are embedded in the {conversation text}, {work order attributes}, and {knowledge document} slots respectively.

[0034] The Deep Thinking Chaining and Answer Generation subunit: Given a verifiable conversation sample, the pre-trained large language model receives the concatenated multi-context input and automatically generates a Deep Thinking chain of thoughts and the final answer based on its internal representation.

[0035] The cross entropy loss function is used to train the subunit. During training, the cross entropy loss function is introduced to minimize the prediction error of the model for the target sequence.

[0036] Furthermore, the reinforcement learning unit for enhancing government thinking specifically includes:

[0037] The decision-making process constructs subunits, and the text sequence generated by the model when thinking and answering is regarded as a decision-making process;

[0038] The reward function design subunit measures the contribution of these decisions to the final goal by designing a reward function that includes two elements: attribute completion and answer matching degree.

[0039] The proximal policy optimization subunit uses the proximal policy optimization (PPO) algorithm to amplify the information correctly completed or corrected in the thinking stage, while incorrect filling or unnecessary retrieval behavior is suppressed.

[0040] Furthermore, the reasoning module includes:

[0041] An input receiving unit receives input of conversation data and work order data;

[0042] Key information extraction unit, extracting key information marked on the work order;

[0043] The deep thinking analysis unit comprehensively analyzes this information through the deep thinking process, triggers knowledge retrieval, and searches the knowledge base for policy documents and similar examples;

[0044] The answer generation unit inputs the retrieved text together with the current context into the model to generate the answer.

[0045] Furthermore, the government question-and-answer system also includes: a verifier module, which is used to verify and score the answers generated by the model in deep thinking reinforcement learning to ensure the accuracy and reliability of the answers.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. The government affairs question and answer system based on deep government affairs thinking provided by the present invention introduces a deep government affairs thinking mechanism by constructing a "domain data pre-training large model + RAG" model, which significantly improves the accuracy and professionalism of government affairs question and answer.

[0048] 2. This invention integrates multi-source government data through the Data Construction Module, optimizes the search structure through the Knowledge RAG Preprocessing Module, and utilizes the Deep Government Thinking Training Module, using a two-stage training process of "supervised fine-tuning + reinforcement learning" to empower the model with deep reasoning capabilities. During the reasoning process, the system proactively analyzes user needs based on work order information, completes missing attributes, accurately retrieves policy documents, and enables collaborative logical reasoning across multiple departments.

[0049] 3. This invention effectively solves problems such as irrelevant answers to hotline work orders and inaccurate references to laws and regulations. It significantly leads in multiple indicators of fact-based and method-solving tasks, reduces manual intervention, improves work order processing efficiency and citizen satisfaction, and enhances government credibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a data construction process framework diagram;

[0051] Figure 2 These are three types of audio text data examples;

[0052] Figure 3 It is a flowchart of a fine-tuning approach to enhancing deep government thinking;

[0053] Figure 4 It is the Prompt instruction template;

[0054] Figure 5 It is a flowchart of the reinforcement learning method to enhance government thinking;

[0055] Figure 6 It is the flow chart of question-answering model algorithm training;

[0056] Figure 7 is the scoring accuracy graph of different validators. DETAILED DESCRIPTION

[0057] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0058] The following is a further description with reference to the accompanying drawings and specific embodiments:

[0059] The present invention provides a government affairs question and answer system based on deep government affairs thinking. Based on the "domain data pre-training large model + RAG" model, it improves the accuracy and professionalism of the question and answer system by introducing deep government affairs thinking in the generation process.

[0060] The specific idea is to let the model conduct a multi-faceted analysis of user needs based on existing work order fields (dialogue, questions, demands, department labels, etc.) before generating answers, and then determine the necessity of retrieving the knowledge base, and make appropriate supplementary inferences for possible ambiguous or missing information. In this way, the government question-and-answer model can more accurately touch upon the subtle differences and potential contexts that are prevalent in government hotline scenarios. By jointly modeling deep thinking in the pre-training stage and the reinforcement learning stage, the model gradually masters the ability to make reasonable inferences in the absence of labels or incomplete work orders, and can effectively integrate external resources such as policies and regulations, historical question-and-answer examples, etc. when answering, thereby producing more professional, easy-to-explain and consistent responses.

[0061] 1.1 Analysis and Construction of Work Order Business Support Data Elements

[0062] The overall dataset construction framework is as follows Figure 1 As shown in the figure, it includes raw data, dataset construction methods, target data, and knowledge RAG preprocessing methods. This figure illustrates the overall process of generating target data such as conversation requests, responses, and corresponding labels from multi-source data such as conversation audio, work order text, and knowledge documents through steps such as ASR models, data screening algorithms, and multi-round self-feedback classification networks. It also constructs a knowledge base and RAG search mechanism to provide retrieval and answer support for deep thinking-driven government large-scale models.

[0063] 1.1.1 Factor Analysis of Government Data

[0064] When building a deep-thinking-driven government model retrieval and generation framework, the quality and diversity of data elements are crucial to the model's deep reasoning and generation performance. Based on the hotline's business characteristics and data collection, data can be divided into three major categories: knowledge documents, conversation recordings, and work order text data. Multi-dimensional correlation is then used to achieve high-quality model training and testing. The following section analyzes each data element and its role in the framework.

[0065] Knowledge document data: The knowledge base data primarily consists of knowledge documents and question-and-answer (QA) records. Its core elements include the knowledge <document number>, <title>, <keywords>, <department>, and corresponding <questions> and <answers> (QA section). The hotline's knowledge base is timely and dynamically expanded. New policies, regulatory interpretations, and frequently asked questions are continuously accumulated on the platform, forming a rich and diverse text corpus. These knowledge documents and corresponding Q&A can be used by large models to contextually embed knowledge from external RAGs during the deliberation process, supporting the model's specific knowledge retrieval, understanding, and reasoning in government scenarios.

[0066] Conversation recordings: Real-time communication records between agents and citizens. To ensure data quality, this application conducts preliminary screening of the recordings when selecting target data, removing unclear, duplicate, or empty recordings. It also performs speech transcription and anonymization. While the text of the conversation recordings overlaps with the work order content, they contain more semantic details and interactive processes. Work order recordings are primarily used to construct conversation embeddings for the command prompt template.

[0067] Work order text data: Work order text data is an important text resource supporting this framework. The hotline generates a huge number of work orders every day. Each work order text contains attribute element information such as <title>, <content>, <label>, <handling department> and <reply>. The work order samples selected in this application focus on various types such as policy guidance, legal consultation, and processing suggestions. This type can reflect the diverse needs of the public in the process of government services. The <label> element can be supplemented by the work order classification model. The work order text data is mainly used to construct the work order attribute embedding of the instruction context prompt template.

[0068] Work order text data can be linked to corresponding conversation recordings to form a complete semantic chain. By integrating the mapping relationship between work order text and conversation recordings, it is possible to provide comprehensive information context for large models during the fine-tuning phase, enabling deep logical reasoning and accurate cross-domain knowledge retrieval.

[0069] 1.1.2 Dataset Construction

[0070] To build a high-quality dataset, this application comprehensively utilizes three types of raw data: conversation recording data, work order text data, and knowledge document data.

[0071] First, the conversation recording data is pre-processed by the ASR (Automatic Speech Recognition) model, which includes three sub-models: language recognition, speaker recognition, and speech recognition. The process is as follows: Using the CAM++ language recognition model

[82] After distinguishing between Mandarin and Cantonese recordings, CAM++ and clustering algorithms were used to classify the recordings.

[82] The combined speaker verification model realizes the speaker identity distinction between agents and citizens, and finally adopts the Paraformer-large long audio model

[83] Convert speech to text.

[0072] The above models are from open-source industrial-grade speech-to-text AI models, all of which have excellent recognition accuracy. The models have high accuracy in recording environments with high clarity and low interference, and can obtain relatively complete conversation texts.

[0073] The resulting text was then subjected to multiple screenings: non-Mandarin, blank, or low-information text was excluded, as well as audio recordings related to "return visits" topics. Considering the redundancy of modal particles, low-entropy paragraphs, and some recognition errors in the actual collection, this application deployed the Qwen2.5-72B-Instruct large model locally to extract the core "appeal-response" elements of the text, and converted and reorganized it according to Alpaca's instruction-supervised fine-tuning format, and finally stored it in JSON format.

[0074] For the work order text, this application uses a multi-round self-feedback classification network to combine and judge its title and content to identify labels such as department and business type. To ensure text purity, pre-processing removes common templated expressions such as "Hello, you reported through XXX channel..." and "[Mayor's Express, warm and worry-free]" and removes duplicate and blank information to obtain the target data corresponding to the work order.

[0075] Knowledge document data covers a variety of types, including notices, business guidelines, and service guides. This framework integrates these documents, uses a deduplication algorithm based on text similarity matching, and performs coarse-grained text segmentation. Finally, it saves the data in CSV format for easy subsequent retrieval and training.

[0076] To assess the model's ability to understand and respond to user questions in government scenarios, conversational text data was categorized as factual data based on explicit, conclusive keywords such as "can / can / can / can't / is this possible"; questions involving operational instructions such as "how / which situations / how / how to / what to do" were classified as solution data. To further measure the complexity of responses within the solution category, a second categorization was performed based on response length, distinguishing between brief, single-step instructions and complex, multi-step solutions involving cross-departmental collaboration.

[0077] Figure 2 Examples of factual data and solution-based data are presented. Finally, 6,000 records were randomly selected from the audio transcription results for classification. This was used to construct test sample data and validate the capabilities of the deep thinking-driven government model retrieval and generation framework. Detailed data statistics are shown in Table 1. This dataset construction method balances text quality and task diversity, providing a solid foundation for subsequent model training, inference, and application in real-world government scenarios.

[0078] Table 1 Detailed statistics of 6000 test samples

[0079]

[0080] 1.1.3 Knowledge RAG Preprocessing

[0081] In order to efficiently support subsequent knowledge retrieval and answer generation, the knowledge RAG preprocessing stage mainly cleans, segments and indexes the work order text data D and knowledge document data K.

[0082] D={d1,d2,...,d M},K={k1,k2,...,k N}(4.1)

[0083] where d i represents the i-th work order text, kj Represents the jth knowledge document. After preliminary cleaning, the text content will be segmented according to the knowledge items to control the length and semantic integrity of the text block. Assuming the maximum segment length is L words and the overlap length is O words, for each text t∈(D∪K), it can be split into several continuous and partially overlapping sub-segments

[0084]

[0085] Each of these s i The length of the word does not exceed L, and adjacent segments have O-shaped overlaps at the boundaries to maintain semantic cohesion. After the segmentation is completed, each sub-segment will be fine-grained lexicalized through the Jieba word segmentation tool and keyword extraction. In this process, a vocabulary can be defined

[0086] V={w1,w2,...,w |V|}, (4.3)

[0087] where w k Represents a segmentation word or keyword. The vocabulary is mapped to the text position of the corresponding sub-segment for subsequent retrieval operations. To quickly build a retrieval structure based on keywords and segmentation results, the Elasticsearch search engine uses an inverted index scheme to reversely map the keywords of each sub-segment to the paragraph and specific position to form an index library. Through the inverted index:

[0088]

[0089] It can make it possible to accurately retrieve data containing a certain keyword w in the query stage. k Text snippets i and its corresponding p in the paragraph i,k This mechanism greatly improves retrieval efficiency, especially when processing large amounts of text, and can quickly filter out irrelevant content and locate the target paragraph.

[0090] In addition, segmented text can optionally be vectorized, generating word or sentence vectors for each subsegment and storing them in a vector index. This vectorization approach can use cosine similarity or Euclidean distance to measure the similarity between texts, further improving the ability to recall semantically related content. For applications requiring more precise, paragraph-level semantics, a multi-search strategy combining a lexical inverted index and a vector index can be used to achieve precise and comprehensive searches.

[0091] Based on the above preprocessing process, each sub-segment is cleaned, segmented, tokenized, and indexed. In the subsequent "knowledge retrieval" stage, the text fragment closest to the query intent can be quickly retrieved based on the user's question or conversation context, and further embedded into the prompt to complete the answer generation combined with the large model.

[0092] 1.2 Training Framework for Deep Political Thinking

[0093] Given a question q and a ticket knowledge base in Representation Knowledge Base The goal of this application is to develop a framework S that can use the knowledge base The framework should automatically identify and parse key information in the ticket to provide effective solutions or answers to support the needs of service agents or internal technical support.

[0094] 1.2.1 Overview of the overall framework

[0095] This application proposes a Deep Governance Thinking Training (DGTT) framework, which empowers large models with deep government thinking capabilities, thereby completing government question answering.

[0096] Hotline ticket data includes tickets with complete attributes, as well as a large number of tickets with missing attributes. To address these two types of ticket data, the Deep Government Thinking training framework is divided into two phases of training. The first phase is to enhance the fine-tuning of Deep Government Thinking. The goal is to fine-tune the large model base using ticket data with complete attributes, transforming ticket information into intermediate results of thinking (the process of Deep Government Thinking). The final result of the ticket processing is used to fine-tune the large model. The second phase is to enhance reinforcement learning for Deep Government Thinking. The goal is to use reinforcement learning methods to enhance the large model's ability to perform deep thinking when handling tickets with incomplete attributes.

[0097] During the inference phase, the government Q&A model first receives input from conversational and work order data. It extracts key information from the annotated work order, such as the request, question, request, tags, and department. It then comprehensively analyzes this information through a deep deliberation process. This deliberation phase primarily simulates the real-world workflow of agents, including determining the type of user need and assessing potential attribute gaps or inconsistencies. It also triggers knowledge retrieval, searching the constructed knowledge base for policy documents and similar examples. The retrieved text, along with the current context, is then fed into the model to generate an answer.

[0098] At the output level, the model will give the final answer, the first stage uses supervised fine-tuning method training, and the second stage uses reinforcement learning method training. By explicitly implanting a deep thinking link before generation, the model can more effectively capture the fine-grained semantics in real scenarios and accurately call external documents, thereby significantly enhancing the explainability and practicality of the answer.

[0099] 1.2.2 Fine-tuning method for enhancing deep government thinking

[0100] The first stage of fine-tuning method for enhancing deep government thinking aims to enable large models to master the ability to comprehensively analyze government work order information and actively "think" or reason before answering. Supervised fine-tuning training includes detailed dialogue content, explicitly stated questions and demands, possible corresponding label information, and document references matching the knowledge base, as shown in Figure 3 If part of the work order has recorded the processing department and department reply, it can be considered as an ideal high-quality supervision signal.

[0101] In this method, first, the large model uses dialogue data to retrieve the knowledge base, and the unified context Prompt instruction template is constructed for the multi-source information of government work orders. The dialogue text, structured attributes, and external knowledge documents are embedded into slots such as {dialogue text}, {order attributes}, and {knowledge documents}, as shown in Figure 4 .

[0102] Given a verifiable dialogue sample can be considered as a tuple (x, y * ), where x represents the complete user demand dialogue sequence, and y * is the confirmed dialogue reply. In the execution phase, the pre-trained large language model receives the spliced multi-context input and automatically generates a deep thinking chain of thought e and the final answer y based on internal representation.

[0103] Specifically, let x represent the dialogue embedding, i.e., embedding the demand part of the dialogue data into the {dialogue text} placeholder; let a represent the attribute embedding, embedding the structured attributes such as order labels, department information, or question types into {order attributes}; let k represent the knowledge embedding, which is generated by retrieving the query generated by the large model, and let the knowledge base be First, perform retrieval in using dialogue embedding x, and get knowledge embedding k. During the retrieval process, it is limited by the label attributes of a (for example, x is "What procedures are needed to open a bakery?", and the label attribute of a is "enterprise operation / license and approval / accommodation and catering industry", then the knowledge retrieval range is limited to the specified relevant knowledge type and similar cases with the same label):

[0104]

[0105] The search results are embedded in the {knowledge document} placeholder. Subsequently, x, a, and k are combined with the model prompt template to form an expanded input sequence X. During reasoning, the model triggers chain thinking e based on the diverse information within X, sequentially deducing the integration process of conversation semantics, attribute content, and external knowledge, ultimately generating the answer y.

[0106] X=<x,a,k> (4.6)

[0107] (e,y)~P θ (e,y|X) (4.7)

[0108] During training, the cross entropy loss function is introduced to minimize the model's prediction error for the target sequence, which is denoted as

[0109]

[0110] Among them, P θ (·) is the large model with θ as parameter for word y at the current time step t The conditional probability estimate of . This multi-context embedding framework explicitly invokes the "deep thinking" phase during reasoning via the Prompt instruction, allowing the model to write out e in the internal sequence before generating the final answer, reflecting the chained reasoning logic integrating x, a, and k. During global training, minimizing the cross-entropy loss function gradually adapts the model to comprehensively infer the triple information of {conversation text}, {work order attributes}, and {knowledge documents}, thereby generating consistent and verifiable generation results from multiple sources in government work order scenarios.

[0111] 1.2.3 Reinforcement Learning Methods to Enhance Government Thinking

[0112] In government practice, many work orders cannot ensure that all information fields are filled in completely and correctly. For example, the user may only describe part of the request, or the work order label is missing or even incorrectly labeled. If it only relies on the aforementioned pre-training based on complete data, the model often finds it difficult to automatically identify and complete these missing links in real-world scenarios. After completing the first stage of supervised fine-tuning, the model has acquired a certain degree of complex reasoning ability. Next, this application proposes a reinforcement learning method to enhance government thinking to further improve this ability, such as Figure 5 shown.

[0113] Conversation data is used to construct instruction templates and knowledge retrieval. In order to introduce reinforcement learning under the deep thinking framework, the model can actively make reasonable inferences in the "thinking" stage to improve its adaptability to missing attributes or labels, and perform necessary self-inspection and error correction when answering. The template only provides two slots: {Conversation Request} and {Knowledge Document}, and requires incomplete work order attributes in the instructions.

[0114] The text sequence generated by the model when thinking and answering is regarded as a decision-making process, and the contribution of these decisions to the final goal is measured by designing a reward function. Let the output sequence generated by the model based on the input X be z, where z is the concatenation of the thinking part and the final answer part, and obeys the model's policy distribution π θ (z|X). The reinforcement learning goal can be expressed as the expected return maximization problem:

[0115] z=<x,a>(4.9)

[0116]

[0117] Among them, R(z) is a reward function used to measure the performance of the generated sequence z in multiple dimensions. Combined with the government scenario, the reward function can be constructed as a weighted sum of the following two types of factors:

[0118] 1. Attribute completion: If the model detects that certain ticket fields (such as tags or claims) are missing during the thinking process and gives a logical filling value, it will be given a positive reward; if it does not fill in the value or fills in the value incorrectly, it will be given a negative reward.

[0119] 2. Answer Matching: If the model's answer matches existing annotations or historical ticket answers, a positive reward is given. If the answer only partially overlaps in key points, a moderate reward is given. If the answer is completely off-topic or inconsistent with the actual record, a negative reward is given.

[0120] Given the prevalence of aliases in the government sector, the exact matching method commonly used in mathematics is not applicable. This application uses a locally deployed large model as a validator, and the provided text and judgment conditions prompt it to return a reward score. y is the answer output by the model, y * The reference answer to the work order defines two sub-reward functions, attribute completion reward r attr (e,a) and answer matching reward r ans (y,y * ), the formula is as follows:

[0121]

[0122] Finally, the two are combined in a weighted manner to obtain the total reward function R(z):

[0123] R(z)=α·r attr (e,a)+β·r match (y,y * )(4.13)

[0124] Where α and β are adjustable weight hyperparameters. Using the Proximal Policy Optimization (PPO) algorithm, the correctly completed or corrected information in the thinking phase is amplified, while incorrect completion or unnecessary retrieval behavior is suppressed.

[0125] By iteratively training on a large amount of incomplete and even noisy work order data, the model can gradually learn how to make optimal decisions in the thinking phase, including when to call on external knowledge and how to correct incorrect labels.

[0126] 1.3 Algorithm Training Process

[0127] Combining the ideas of deep thinking pre-training and reinforcement learning, the specific model training process can be summarized into the following 5 steps: Figure 6 As shown:

[0128] Step 1: Data filtering and formatting

[0129] Collect conversation text and work order attributes x,a from the training dataset, uniformly construct them into multi-context input and perform preliminary cleaning to prepare data for subsequent in-depth thinking.

[0130] Step 2: Knowledge RAG and build prompt template

[0131] According to x, a uses the thinking ability of the large model to extract the external knowledge document k, embed the context prompt instruction template, and obtain the input sequence X

[0132] Step 3: Fine-tuning training on deep government thinking

[0133] Perform supervised SFT training on the obtained formatted data to minimize the cross entropy loss The answer area in , which is considered part of the target sequence.

[0134] Step 4: Optimizing reinforcement learning strategies for deep government thinking

[0135] After fine-tuning in the first phase of Step 3, for work orders with missing or inconsistent fields, a reinforcement learning promp template is constructed to allow the model to supplement the missing attributes in the generated thinking, and then use the reward algorithm R(z) to score its output, and then pass To guide the model to continuously iteratively update.

[0136] Step 5: Output Fusion

[0137] During the inference phase, after receiving a new ticket, the model first begins a process of reflection based on the conversation, automatically invoking the RAG search knowledge base. After the search is complete, the model fuses the reflection results with the document information to generate a final answer.

[0138] 1.4 Experiment and Analysis

[0139] 1.4.1 Dataset Introduction

[0140] The dataset for this application is derived from the corpus resources of the hotline, which are used for fine-tuning and strengthening training samples, as well as 6,000 test samples. The data samples are divided into three types: "fact type", "method solution type (short reply)" and "method solution type (long reply)". In order to ensure the balance between training and evaluation, 500 samples are extracted from each of the three types of data in the training set, totaling 1,500 samples as a validation set with a fixed number of steps. Ensuring coverage of various question forms enables the model to obtain a more stable performance estimate in terms of fact answering and solution generation.

[0141] 1.4.2 Evaluation indicators

[0142] For factual data, this application uses the widely used EM (ExactMatch) and F1 metrics in the fact evaluation task to assess the consistency achieved by this application's model in generating accurate answers to factual queries. For method solution data, this application uses BLEU and ROUGE metrics to evaluate the content quality of sentences generated by the model.

[0143] EM measures whether the answer predicted by the model is completely consistent with the reference answer. If the model's answer is exactly the same as the reference answer, the EM score is 1, otherwise it is 0.

[0144] F1 is the harmonic mean of Precision and Recall. In question answering tasks, the F1 score is often used to evaluate the degree of partial match between the model's predicted answer and the ground truth answer.

[0145] BLEU is generally used to evaluate the quality of machine translation. This metric reflects the accuracy and fluency of the translation by counting the overlap between the candidate text and the reference text at the n-gram level. For government affairs, BLEU is used to characterize the readability and closeness of the response sentence in terms of language expression.

[0146] ROUGE is primarily used to evaluate automatic text summarization. ROUGE calculates the n-gram overlap between generated text and reference text. This metric, introduced in this application's solution generation evaluation, can, to a certain extent, reflect whether the key information in the response is complete while also meeting recall requirements.

[0147] 1.4.3 Baseline Method and Experimental Setup

[0148] The baseline methods used in this application can be divided into two categories: one is the open-source open-domain question answering and retrieval enhancement generation scheme, and the other is the modern method of integrating knowledge RAG based on open-source large models. These methods are all around open-domain question answering or retrieval enhancement generation, and through the injection of retrieval fragments in the input information or the construction of document-level classifiers, etc., to improve the reasoning and answering accuracy of language models.

[0149] • In-Context RALM: This paper proposes a retrieval-augmented language model method called In-Context RALM, which does not need to modify the language model architecture, but only by splicing relevant retrieval documents in front of the input text to improve the performance and accuracy of language model generation, while developing a document reordering mechanism specifically for language modeling.

[0150] • RfiD: This research proposes an open-domain question answering model called RFiD, which adds a document-level classifier on top of the encoder of the RFiD model to explicitly distinguish between rational documents containing answers and irrelevant documents, and guides the decoder to generate answers through rationalized embeddings, thereby reducing the interference of false associations and improving the reasoning ability of the model.

[0151] • GLM-4-9B-Chat+RAG: GLM-4-9B-Chat is the latest pre-training model GLM-4 series released by Zhi Spectrum AI, which supports multi-round dialogue, web browsing, code execution, custom tool invocation and long text reasoning. This baseline integrates external knowledge RAG retrieval.

[0152] • Qwen2.5-7B-Instruct+RAG: Qwen2.5-7B-Instruct is a 70 billion parameter language model developed by Alibaba, which has strong language understanding and generation capabilities after instruction fine-tuning, supports multi-language processing, and is suitable for various natural language processing tasks. This baseline integrates external knowledge RAG retrieval.

[0153] • DeepSeek-R1-Distill-Qwen-7B+RAG: DeepSeek-R1-Distill-Qwen-7B is a 70 billion parameter language model developed by DeepSeek, a Chinese artificial intelligence company. This model is based on the Qwen model and fine-tuned through the distillation of DeepSeek-R1 knowledge, aiming to improve reasoning ability and multi-task processing performance. This baseline integrates external knowledge RAG retrieval.

[0154] The DeepSeek-R1-Distill-Qwen-7B is used as the model base of the DGTT method, and three significant open source large language models are subjected to a general supervised fine-tuning process, and all experiments are carried out in a high-performance environment of 4 NVIDIA A800-SXM4-80GB GPUs. The fine-tuning process of all large models uses the LoRA method, with a rank size of 8, a scaling factor of 16, a default learning rate of 5e-5, a batch size of 8, a training precision of bf16, a validation set evaluation every 500 steps, and a checkpoint saving, and finally 3 epochs are trained for each stage.

[0155] In addition, Qwen2.5-72B-Instruct is deployed locally as a reward model verifier and a claim extractor in knowledge RAG preprocessing. The Top-p sampling value and the temperature coefficient are both fixed at 0.7, and the vllm engine is used for inference acceleration.

[0156] 1.4.4 Experimental results and analysis

[0157] As can be seen from Table 2, the DGTT proposed in the present application performs outstandingly in each index of the fact type and the method solution type (including short and long answers). Compared with In-Context RALM, RFiD and various open source schemes based on Qwen or GLM, DGTT leads in each index by a significant margin.

[0158] Notably, further comparison of the DGTTw / oRLHF version shows that reinforcement learning with multi-dimensional rewards can effectively improve the accuracy (EM, F1) and generation quality (BLEU, ROUGE) of answers. The multi-dimensional reward signal based on reinforcement learning can effectively promote the model to deeply think and reason on multi-source information in complex government scenarios, thereby significantly enhancing the explainability and robustness of the model, indicating that the attribute completion and answer matching mechanism can reduce false filling and make full use of prior knowledge to generate answers that better meet real needs.

[0159] Table 2 Comparison of main experimental results

[0160]

[0161] 1.4.5 Ablation experiment

[0162] In order to verify the effectiveness of the proposed method, the present application conducts an ablation experiment, and the results are recorded in Table 3.

[0163] Table 3 Ablation experiment results. w / o means without

[0164]

[0165] As shown in the table, the complete DGTT framework achieved the best performance across all evaluations. However, each time a component was removed, model performance declined to varying degrees. First, "without RLHF" eliminated the reinforcement learning component based on multi-dimensional reward signals, which weakened the model's ability to correct attribute completion and retrieval content, resulting in a relatively lower overall accuracy and generation quality. Furthermore, "without RLHF+DeepThinking" also eliminated the deep thinking mechanism, making it difficult for the model to perform multi-round chained reasoning and key information mining, resulting in an even more significant performance degradation. "without RLHF+DeepThinking+SFT" and "without RLHF+DeepThinking+RAG" further eliminated fine-tuning training or retrieval augmentation, making it impossible for the model to effectively utilize external knowledge to corroborate answers or fill in missing elements, further reducing the completeness and interpretability of answers. Finally, "without RLHF+DeepThinking+RAG+SFT" completely stripped out supervised fine-tuning and external retrieval support, relying solely on the base model for answering, significantly impacting accuracy and consistency. Overall, there is a synergistic gain relationship between components such as reinforcement learning, deep thinking, and retrieval enhancement. The lack of any one of them will weaken the model's comprehensive utilization and accurate generation of multi-source information in government scenarios.

[0166] Verifiers play an important role in deep learning reinforcement learning (RL). In this application, Qwen2.5-72B-Instruct is used as a verifier. To evaluate its reliability, this application manually verified 200 scoring instances and compared the accuracy of attribute matching rewards and answer rewards in various parameter reward models in judging the quality of large model generation. Figure 7 As can be seen, Qwen2.5-72B-Instruct achieved the highest accuracy in both answering and attribute verification; Qwen2.5-32B-Instruct and Qwen2.5-7B-Instruct declined slightly, while ExactMatch achieved the lowest accuracy. A comparison based on annotated examples shows that large-parameter models are more reliable in judging text details and attribute fields, significantly reducing missed and false positives, while smaller-parameter models still lag behind rule-based judgment.

[0167] To sum up, the government question and answer system based on deep government thinking provided by the application significantly improves the accuracy and professionalism of government question and answer by constructing a "domain data pre-training large model + RAG" mode and introducing a deep government thinking mechanism. The application integrates multi-source government data through a data construction module, optimizes the retrieval structure through a knowledge RAG preprocessing module, and uses a deep government thinking training module to give the model deep reasoning ability through two-stage training of "supervised fine-tuning + reinforcement learning". In the reasoning process, the system can actively analyze user demand based on work order information, complete missing attributes, accurately retrieve policy documents, and realize multi-department collaborative logical reasoning. Compared with the prior art, the application effectively solves the problems of "answering irrelevant questions" of hotlines and inaccurate regulations, significantly leads in multiple indicators of fact and method solving tasks, reduces manual intervention, improves work order processing efficiency and citizen satisfaction, and enhances government credibility.

[0168] The above is only the preferred embodiment of the application and is not used to limit the application. For those skilled in the art, the application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A government affairs question-answering system based on deep government affairs thinking, characterized by: include: The data construction module is used to generate high-quality government Q&A datasets from multi-source data through data preprocessing, screening, and classification; The knowledge RAG preprocessing module is used to clean, segment, and index work order text data and knowledge document data to support subsequent knowledge retrieval and answer generation; The deep government thinking training module empowers large models with deep government thinking capabilities through fine-tuning and reinforcement learning, enabling them to answer government questions. The reasoning module is used to receive inputs of conversation data and work order data, extract key information, comprehensively analyze this information through a deep thinking process, trigger knowledge retrieval, search for policy documents and similar instances in the retrieval knowledge base, and then input the retrieved text together with the current context into the model to generate answers.

2. A government affairs question-answering system based on deep government affairs thinking according to claim 1, characterized in that: The data construction module includes: The data element analysis unit divides data into three categories: knowledge documents, conversation recordings, and work order text data, and achieves high-quality training and testing of the model through multi-dimensional association; The dataset construction unit comprehensively utilizes conversation recording data, work order text data, and knowledge document data, and generates high-quality datasets through multiple screening and preprocessing.

3. A government affairs question-and-answer system based on deep government affairs thinking according to claim 2, characterized in that: The knowledge document data includes knowledge <document number>, <title>, <keywords>, <department> and corresponding <questions> and <answers>; the conversation recording data is pre-processed by the ASR model and used to construct the conversation embedding of the instruction prompt template; the work order text data contains attribute element information of <title>, <content>, <label>, <processing department> and <reply>, which is used to construct the work order attribute embedding of the instruction context prompt template.

4. The government affairs question-answering system based on deep government affairs thinking according to claim 2 is characterized in that: The data set construction unit specifically includes: The conversation recording data processing sub-unit, after pre-processing with the ASR model, excludes non-Mandarin, blank, or low-information text, as well as recordings related to "return visits." It then extracts the core "appeal-response" elements of the text and converts and restructures it according to Alpaca's instructions and fine-tuning format. The work order text data processing sub-unit uses a multi-round self-feedback classification network to combine and judge the title and content, identify the department and business type labels, delete common template expressions, and eliminate duplicate and blank information; The knowledge document data processing sub-unit covers notices, business guidelines and service guides. It adopts a deduplication algorithm based on text similarity matching, performs coarse-grained segmentation on the text, and finally saves it in CSV format.

5. The government affairs question-answering system based on deep government affairs thinking according to claim 2 is characterized in that: The knowledge RAG preprocessing module includes a knowledge RAG preprocessing unit that cleans, segments, and constructs indexes for work order text data and knowledge document data; The knowledge RAG preprocessing unit specifically includes: The text segmentation sub-unit segments the text content according to knowledge items to control the length and semantic integrity of the text blocks; Lexicalization subunit: Each sub-segment is lexicalized in a fine-grained manner using the Jieba word segmentation tool and keyword extraction. A vocabulary is defined and mapped to the text position of the corresponding sub-segment. The index construction sub-unit uses the inverted index scheme of the Elasticsearch search engine to reversely map the keywords of each sub-segment to the paragraph and specific location to form an index library; The vectorization processing subunit optionally introduces vectorization processing to the segmented text, generates word vectors or sentence vectors, and stores them in the vector index.

6. The government affairs question-answering system based on deep government affairs thinking according to claim 1 is characterized in that: The in-depth government affairs thinking training module includes: Enhance the fine-tuning unit of deep government affairs thinking, fine-tune the large model base through work order data with complete attributes, and convert work order information into intermediate results of thinking; The reinforcement learning unit that enhances deep government thinking uses reinforcement learning methods to enhance the deep thinking ability of large models when processing work orders with incomplete attributes.

7. A government affairs question-answering system based on deep government affairs thinking according to claim 6, characterized in that: The fine-tuning units for enhancing in-depth government affairs thinking specifically include: The context prompt instruction template constructs a sub-unit. The large model uses conversation data to search the knowledge base and build a unified context prompt instruction template. The conversation text, structured attributes, and external knowledge documents are embedded in the slots of {conversation text}, {work order attributes}, and {knowledge document} respectively. The Deep Thinking Chaining and Answer Generation subunit: Given a verifiable conversation sample, the pre-trained large language model receives the concatenated multi-context input and automatically generates a Deep Thinking chain of thoughts and the final answer based on its internal representation. The cross entropy loss function is used to train the subunit. During training, the cross entropy loss function is introduced to minimize the prediction error of the model for the target sequence.

8. The government affairs question-answering system based on deep government affairs thinking according to claim 6 is characterized in that: The reinforcement learning unit for enhancing government thinking specifically includes: The decision-making process constructs subunits, and the text sequence generated by the model when thinking and answering is regarded as a decision-making process; The reward function design subunit measures the contribution of these decisions to the final goal by designing a reward function that includes two elements: attribute completion and answer matching degree. The proximal policy optimization subunit uses the proximal policy optimization (PPO) algorithm to amplify the information correctly completed or corrected in the thinking stage, while incorrect filling or unnecessary retrieval behavior is suppressed.

9. The government affairs question-answering system based on deep government affairs thinking according to claim 1 is characterized in that: The reasoning module includes: An input receiving unit receives input of conversation data and work order data; Key information extraction unit, extracting key information marked on the work order; The deep thinking analysis unit comprehensively analyzes this information through the deep thinking process, triggers knowledge retrieval, and searches the knowledge base for policy documents and similar examples; The answer generation unit inputs the retrieved text together with the current context into the model to generate the answer.

10. The government affairs question-answering system based on deep government affairs thinking according to claim 1 is characterized in that: The government affairs question-and-answer system also includes: a verifier module, which is used to verify and score the answers generated by the model in deep thinking reinforcement learning to ensure the accuracy and reliability of the answers.

Images