Intelligent government affair-oriented large model multi-modal government affair data intelligent retrieval and analysis system

The multimodal government data intelligent retrieval and analysis system solves the semantic heterogeneity problem of multimodal government data, realizes accurate identification of user intent and generation of evidence chains, meets the requirements of government security and compliance, and improves the accuracy and credibility of government data retrieval.

CN123068445APending Publication Date: 2026-11-10AN HUI CHANG JIANG SHU ZHI YUN JI SUAN KE JI GU FEN YOU XIAN GONG SI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611093064.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-22
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing government retrieval and analysis systems cannot effectively and uniformly process multimodal government data, lack semantic mapping, cannot accurately identify user intent, lack traceable evidence chains, have insufficient access control, and are unable to meet government security and compliance requirements.

Method used

The system employs a multimodal government data access module, a government semantic governance module, a multimodal semantic encoding module, an intent recognition and task routing module, a hybrid retrieval module, an evidence chain analysis module, and an access control and auditing module to achieve multimodal data preprocessing, semantic encoding, intent recognition, multi-channel retrieval, and evidence chain generation. It also generates government responses by combining a large language model and performs access control and auditing.

Benefits of technology

It achieves unified semantic understanding of multimodal government data, accurately identifies user intent, generates traceable evidence chains, meets government security and compliance requirements, and improves retrieval accuracy and result credibility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068445A_ABST
    Figure CN123068445A_ABST
Patent Text Reader

Abstract

The application relates to the technical field of smart government data processing, and discloses a large model multi-modal government data intelligent retrieval and analysis system for smart government, wherein the system comprises a multi-modal government data access module, a government semantic management module, a multi-modal semantic coding module, an intention recognition and task routing module, a hybrid retrieval module, an evidence chain analysis module, a large model generation and verification module, and a permission control and auditing module. The system uniformly codes multi-modal government data to a government semantic space, generates traceable government replies through multi-channel hybrid retrieval and structured evidence chain organization, and performs permission control and behavior trace throughout the whole link.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of smart government data processing technology, and more specifically, to a large-scale multimodal government data intelligent retrieval and analysis system for smart government. Background Technology

[0002] In the field of smart government, government data is characterized by multi-source heterogeneity, multi-modal coexistence, cross-departmental dispersion, and frequent updates. Government data includes both structured data (such as item lists, approval flow records, basic population and enterprise databases, regulatory ledgers, case handling logs, and performance reports) and unstructured or semi-structured data (such as policy documents, notices and announcements, meeting minutes, service guides, law enforcement documents, scanned images, audio and video interviews, hotline recordings, and form screenshots).

[0003] Existing government affairs retrieval and analysis systems typically employ a single retrieval method based on keyword matching, combined with limited structured database query capabilities, to respond to user-submitted government affairs query requests and directly output query results in text form.

[0004] However, the aforementioned existing technologies have the following technical problems: First, different modalities of government data lack a unified semantic mapping; form content in images, service requests in voice recordings, and implicit clauses in scanned documents cannot be retrieved and associated in a unified semantic space. Second, user queries are often colloquial, contextual, and vague; relying solely on keyword recall cannot accurately identify the true intent and multi-dimensional constraints. Third, policy documents and service guides are frequently adjusted; systems lacking dynamic version comparison and timeliness verification are prone to outputting outdated content. Fourth, existing systems only provide conclusions without providing a chain of evidence, failing to meet the requirements of administrative management and supervision for traceability of results. Fifth, a single search method cannot simultaneously address semantic understanding and structured filtering, leading to incomplete recall or insufficient accuracy. Sixth, there is a contradiction between the sensitivity and hierarchical authorization requirements of government data and intelligent retrieval; existing systems lack sufficient granularity in terms of permission filtering, desensitized display, and audit logging, failing to meet government security compliance requirements. Summary of the Invention

[0005] This invention provides a large-scale multimodal government data intelligent retrieval and analysis system for smart government affairs, which solves the technical problems in related technologies such as the diversity of government data modalities, semantic heterogeneity, lack of traceable evidence chains in retrieval results, and difficulty in ensuring the consistency between access control and generated content.

[0006] This invention discloses a large-scale multimodal government data intelligent retrieval and analysis system for smart government affairs, comprising: a multimodal government data access module, configured to access raw government data of multiple modalities, perform preprocessing for different modalities, and encapsulate the preprocessing results into a unified standard intermediate representation; a government semantic governance module, configured to perform data cleaning, deduplication, anonymization, structured extraction, entity recognition, relation extraction, time normalization, and version labeling on the standard intermediate representation to generate government knowledge units and government entity relationship networks; a multimodal semantic encoding module, configured to generate basic semantic vectors for different modal content in the government knowledge units through corresponding encoding branches, and map the basic semantic vectors of each modality to a unified government semantic space through cross-modal alignment to generate a unified government semantic vector and store it in a vector index library; and an intent recognition and task routing module, configured to perform modality discrimination and intent classification on user input. The system decomposes user queries into structured query descriptions containing intent categories, subject information, condition sets, target types, and constraint sets, and generates a task routing plan. A hybrid retrieval module is configured to perform multi-channel hybrid retrieval based on the task routing plan, integrating keyword recall, vector recall, structured filtering, graph association, and timeliness filtering to obtain a set of candidate results. An evidence chain analysis module is configured to perform association sorting, source verification, version comparison, and context completion on the candidate result set, binding each conclusion to at least one source piece of evidence and outputting a structured evidence chain. A large model generation and verification module is configured to generate government responses based on the structured evidence chain using a large language model, and perform strong verification of key information, source verification, and consistency verification on the generated results. An access control and auditing module is configured to perform access filtering during the retrieval phase, provide dynamic desensitization rules during the generation phase, and perform behavior recording during the output phase.

[0007] Furthermore, the multimodal government data access module includes: a text preprocessing submodule, which performs segmentation processing, noise removal, and encoding standardization on text data according to chapter structure and clause numbering to obtain a standardized text paragraph sequence; a table preprocessing submodule, which performs field recognition, row and column alignment, and format standardization on table data to obtain structured table data; an image preprocessing submodule, which performs layout analysis, text region positioning, and optical character recognition on image data and scanned documents to obtain structured image description data; an audio and video preprocessing submodule, which performs speech transcription, key segment segmentation, and timestamp alignment on audio and video data to obtain a time-stamped transcribed text sequence; and a process log preprocessing submodule, which performs status extraction, event serialization, and time chain organization on process log data to obtain a structured event sequence; each of the aforementioned standard intermediate representations includes the original data identifier, modality type label, preprocessed content body, and source metadata.

[0008] Furthermore, in the government semantic governance module: the entity recognition and relationship extraction submodule performs named entity recognition and relationship extraction on the content body of the standard intermediate representation. Entity types include item name, department name, policy file name, material name, processing conditions, time expression, regional name, personnel role, and monetary value. Relationship types include semantic relationships between items and required materials, items and processing departments, items and applicable conditions, policy documents and stipulated items, and policy documents and repealed policy documents. The government entity relationship network uses items, policy documents, materials, departments, conditions, time limits, and processing results as nodes, and the relationship types as edges. The time normalization and version labeling submodule converts relative time expressions into absolute time intervals, labels multiple versions of the same item or the same policy document with version numbers, and marks the current valid version, historical version, and repeal status. Each government knowledge unit includes attributes such as data source, business department, policy level, item name, applicable object, time validity, regional scope, key conditions, evidence location, related items, and permission level.

[0009] Furthermore, in the multimodal semantic encoding module: the text encoding branch uses a pre-trained text encoder to generate text semantic vectors for text-based government knowledge units; the table encoding branch linearizes structured table data into a text sequence with field labels, and then uses the pre-trained text encoder to generate table semantic vectors; the image encoding branch generates dimension-based semantic vectors for optical character recognition results using the pre-trained text encoder. The text semantic vectors are generated by a pre-trained visual encoder from the original image region, using a dimension of [missing information]. The visual semantic vectors are concatenated and then mapped to a dimension through a linear projection layer. The image-comprehension semantic vector, where The dimension of the text semantic vector. The dimension of the visual semantic vector. To unify the vector dimension of the government semantic space; the audio encoding branch uses the pre-trained text encoder to generate a dimension of [missing value] for the transcribed text content. The audio transcription semantic vector; the process encoding branch concatenates the state information, time information, and processing department information of each node in the structured event sequence into a text description, and then uses the pre-trained text encoder to generate a dimension of The process semantic vector.

[0010] Furthermore, the cross-modal alignment is achieved through multiple modality-specific linear projection layers, each trained using a contrastive learning approach, with the loss function being:

[0011] in and The first The unified government semantic space vector is obtained by mapping the two modalities in a positive sample pair through their respective linear projection layers. The index for positive sample pairs. For cosine similarity, For temperature coefficient, This refers to the number of samples within a batch. This serves as the traversal index for samples within a batch. This represents an exponential function; positive sample pairs are data pairs describing different modalities of the same government affairs content, while negative sample pairs are data pairs describing different government affairs content.

[0012] Furthermore, in the intent recognition and task routing module: the intent recognition submodule uses a pre-trained language model to classify the user query text into intent categories, including policy retrieval, material verification, case comparison, trend analysis, process tracking, and knowledge Q&A. The pre-trained language model is subjected to supervised fine-tuning on government query samples labeled with business intent categories. The complex query decomposition submodule extracts subject information, condition information, target information, and constraint information from the user query text. Subject information includes individuals, enterprises, projects, or matters; condition information includes geographical restrictions, time restrictions, industry restrictions, qualification conditions, amount conditions, and frequency conditions; target information includes policy provisions, material lists, processing paths, case references, anomaly prompts, or statistical analysis; and constraint information includes version requirements, departmental filtering, and permission level output requirements. The task routing submodule specifies the target data range and retrieval channel for retrieval based on the structured query description, and generates a task routing plan containing multiple retrieval subtasks for complex queries involving multiple business intent categories.

[0013] Furthermore, in the evidence chain analysis module: the association ranking submodule calculates a comprehensive relevance score for each government knowledge unit in the candidate result set. :

[0014] in For weight parameters, For vector similarity scores, Scoring is based on keyword hits. To score the correlation between the graphs, Score based on timeliness. For permission matching scores, all scores are normalized to a certain value before calculation. The interval; the graph correlation score is calculated by taking the extended hop count for each government knowledge unit obtained through graph correlation supplementation. reciprocal For directly recalled government knowledge units, the value is 1; the weight parameter is obtained by supervised training on a government query sample set labeled with relevance tags, and is continuously updated based on user feedback; the source verification submodule verifies the validity of the evidence position of each government knowledge unit in the sorted candidate result set, and marks government knowledge units that fail the verification as having a pending source status; the version comparison submodule extracts the differences from multiple versions involving the same policy document or the same service guide, and generates a version difference record containing the changed fields, the content before the change, the content after the change, and the effective time of the change; the evidence chain organization submodule organizes the structured evidence chain according to the structure of conclusion, basis, source, time, version, and scope of application, performs cross-verification when multiple pieces of evidence support the same conclusion, and sorts the evidence in case of conflict by version priority.

[0015] Furthermore, in the large model generation and verification module: the prompt word assembly submodule assembles the conclusion summary, evidence content, source identifier, and scope of application in the structured evidence chain into input prompt words for the large language model. The input prompt words include constraints such as generating responses based solely on the provided evidence content, marking the source file name and clause number of cited policy provisions, marking the confidence level of uncertain content, and prohibiting the generation of factual information not included in the evidence. The rule verification submodule performs strong verification of key information, source verification, and consistency verification on the draft government response. The strong verification of key information compares the policy provision number, amount value, time limit value, and processing conditions in the draft government response with the original content of the corresponding evidence in the structured evidence chain item by item. The source verification checks whether each citation has a corresponding source evidence in the structured evidence chain. The consistency verification checks for contradictory statements and unifies them according to version priority. The confidence level marking submodule marks the confidence level of content paragraphs generated based on low-confidence evidence.

[0016] Furthermore, in the access control and auditing module: the identity authentication and access control submodule obtains the user's identity information and parses the data range that the current user can access, including the scope of accessible business departments, policy levels, access levels, and geographical areas. The access control result is passed to the structured filtering submodule of the hybrid retrieval module. The dynamic desensitization rule submodule generates dynamic desensitization rules based on the user's data access level and the access level of the government knowledge unit. The dynamic desensitization rules are passed to the desensitization display submodule of the large model generation and verification module. The behavior recording submodule performs full-link recording of user query behavior, search hit behavior, evidence citation behavior, result export behavior, and manual correction behavior and stores it in the audit log library. The feedback write-back submodule receives user correction feedback and writes it back to the government semantic governance module to trigger the update verification of the corresponding government knowledge unit. It also receives user evaluation feedback and writes it back to the evidence chain analysis module to adjust the weight parameters in the comprehensive relevance score.

[0017] This invention discloses a large-scale multimodal government data intelligent retrieval and analysis method for smart government, used to execute the aforementioned large-scale multimodal government data intelligent retrieval and analysis system for smart government. The method includes: Step 1, accessing multiple modalities of raw government data through a multimodal government data access module, performing preprocessing for different modalities, and uniformly encapsulating the preprocessing results into a standard intermediate representation; Step 2, performing data cleaning, deduplication, anonymization, structured extraction, entity recognition, relation extraction, time normalization, and version labeling on the standard intermediate representation through a government semantic governance module, generating government knowledge units and a government entity relationship network; Step 3, generating basic semantic vectors for different modalities in the government knowledge units through a multimodal semantic encoding module via corresponding encoding branches, and mapping each modal basic semantic vector to a unified government semantic space through cross-modal alignment, generating a unified government semantic vector and storing it in a vector index library; Step 4, when a user initiates a query, the method uses intent... The identification and task routing module performs modality discrimination and intent classification on user input, decomposes user queries into structured query descriptions, and generates task routing plans. Step 5 involves a hybrid retrieval module that, based on the task routing plan, integrates keyword recall, vector recall, structured filtering, graph association, and timeliness filtering to perform a multi-channel hybrid retrieval, obtaining a set of candidate results. Step 6 involves an evidence chain analysis module that performs association sorting, source verification, version comparison, and context completion on the candidate result set, binding each conclusion to at least one source piece of evidence and outputting a structured evidence chain. Step 7 involves a large model generation and verification module that, based on the structured evidence chain, calls a large language model to generate government responses and performs strong verification of key information, source verification, and consistency verification on the generated results. Step 8 involves an access control and auditing module that performs access filtering, dynamic desensitization, and behavior recording in steps 5 to 7, and triggers updates to government knowledge units and adjustments to retrieval sorting parameters based on user feedback.

[0018] The present invention has the following beneficial effects: I. This invention uses a multimodal government data access module to perform targeted preprocessing on government data of different modalities, such as text, tables, images, scanned documents, audio and video, and process logs, and encapsulates them into a unified standard intermediate representation. This makes subsequent processing no longer limited to a single modality, thereby overcoming the problem that traditional government retrieval systems cannot uniformly understand multimodal government data due to the fragmentation of data types.

[0019] Second, this invention uses a cross-modal alignment submodule in the multimodal semantic coding module to map the basic semantic vectors of different modalities to a unified government semantic space through contrastive learning. This allows queries submitted by users in the form of images, voice, or spoken language to perform similarity calculations with government knowledge units of different modalities, such as text-based policy articles and form-based application records, in the unified government semantic space. This overcomes the problem that the lack of a unified semantic mapping between different modal data makes cross-modal retrieval impossible.

[0020] Third, this invention decomposes user query text into structured query descriptions with multiple dimensions such as subject, conditions, target and constraints through the complex query decomposition submodule in the intent recognition and task routing module. The structured query descriptions are then assigned to the corresponding retrieval channels through the task routing submodule. This enables the system to handle colloquial and fuzzy queries with multiple constraints, thereby overcoming the problem that traditional systems rely solely on keyword matching and cannot accurately identify complex government query intents.

[0021] Fourth, this invention integrates five retrieval methods—keyword recall, vector recall, structured filtering, graph association, and timeliness filtering—through a hybrid retrieval module. This allows for the complementarity of exact matching and semantic matching, structured constraints and unstructured semantics, and direct and indirect associations, thereby overcoming the problem of incomplete recall or insufficient accuracy of a single retrieval method in complex government affairs queries.

[0022] Fifth, this invention binds each conclusion to specific source evidence through the evidence chain analysis module and marks the source identifier, time stamp, version information and scope of application. By cross-verification and version priority sorting of conflicting evidence, the integrity and consistency of the structured evidence chain are guaranteed, thereby overcoming the problem that existing systems only output conclusions without providing evidence chains, resulting in untraceable results.

[0023] VI. This invention performs version labeling and timeliness verification on government knowledge units through the time normalization and version labeling submodule in the government semantic governance module and the timeliness filtering submodule in the hybrid retrieval module, giving priority to retaining the latest valid version and marking it as obsolete, thereby overcoming the problem of the system outputting outdated content due to frequent policy updates.

[0024] VII. This invention, through its access control and auditing module, performs access filtering in the retrieval stage, dynamic desensitization in the generation stage, and behavior recording in the output stage, forming a full-link control system that includes filtering before retrieval, constraints during generation, and traceability after output. This overcomes the problem that traditional intelligent retrieval systems lack granularity in access filtering, desensitization display, and auditing traceability, thus failing to meet the requirements of government security compliance. Attached Figure Description

[0025] Figure 1This is a flowchart of the intelligent retrieval and analysis method for large-scale multimodal government data provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the full-link processing flow of the government affairs intelligent question-and-answer system provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the version relationship and entity association network diagram of government knowledge units provided in the embodiments of the present invention; Figure 4 This is a schematic diagram showing the comparison of the comprehensive relevance scores of the candidate result set provided in the embodiments of the present invention. Figure 5 This is a schematic diagram of the candidate result set sorted by comprehensive relevance score provided in an embodiment of the present invention; Figure 6 This is a schematic diagram showing the comparison between vector recall cosine similarity and comprehensive score provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the multi-channel recall process of the hybrid retrieval module provided in the embodiment of the present invention; Figure 8 This is a schematic diagram of the multimodal semantic coding and cross-modal alignment flowchart provided in the embodiments of the present invention. Detailed Implementation

[0026] Example 1 In the field of smart government, government data is characterized by multi-source heterogeneity, multi-modal coexistence, cross-departmental dispersion, and frequent updates. Government data includes both structured data (such as item lists, approval flow records, population and enterprise databases, regulatory ledgers, case handling logs, and performance reports) and unstructured or semi-structured data (such as policy documents, notices and announcements, meeting minutes, service guides, law enforcement documents, scanned images, PDF attachments, audio and video interviews, hotline recordings, on-site photos, and form screenshots). Existing government retrieval and analysis systems face the following technical challenges: data from different modalities cannot be retrieved and correlated within a unified semantic space, leading to ineffective understanding of form content in images, service requests in voice recordings, and implicit clauses in scanned documents; user queries are often colloquial, scenario-based, and vague, and existing systems rely solely on keyword recall, failing to accurately identify true intent and multi-dimensional constraints; policy documents and service guides are frequently revised, and systems lacking dynamic version comparison and timeliness verification are prone to outputting outdated content; government scenarios demand high reliability of results, and existing systems only provide conclusions without evidence chains, failing to meet administrative management and supervisory review requirements; single retrieval methods struggle to balance semantic understanding and structured filtering, resulting in incomplete recall or insufficient accuracy; and there is a conflict between the sensitivity and hierarchical authorization requirements of government data and intelligent retrieval.

[0027] To address the aforementioned technical issues, this embodiment provides a large-scale multimodal government data intelligent retrieval and analysis system, which consists of a multimodal government data access module, a government semantic governance module, a multimodal semantic encoding module, an intent recognition and task routing module, a hybrid retrieval module, an evidence chain analysis module, a large-scale model generation and verification module, and an access control and auditing module.

[0028] The overall architecture of the large-scale multimodal government data intelligent retrieval and analysis system is as follows: The multimodal government data access module converts the raw government data into a standard intermediate representation and then transmits it to the government semantic governance module. The government semantic governance module performs structured extraction and government knowledge unit annotation on the data and then transmits it to the multimodal semantic encoding module. The multimodal semantic encoding module generates a unified government semantic vector and stores it in a unified government semantic space. When a user initiates a query, the intent recognition and task routing module performs intent parsing and task allocation on the user input. The hybrid retrieval module performs multi-channel retrieval from the unified government semantic space. The evidence chain analysis module performs association sorting and source verification on the candidate result set. The large-scale model generation and verification module generates government responses based on the structured evidence chain. The access control and auditing module performs access control and behavior tracking throughout the entire chain.

[0029] The multimodal government data access module is configured to access raw government data in multiple modalities and convert it into a standard intermediate representation. The multimodal government data access module acquires raw data from sources such as government business systems, archive systems, document libraries, hotline systems, mobile data collection terminals, meeting recording systems, and law enforcement evidence collection terminals, and performs basic preprocessing for different modalities to transform the raw data into a unified data packet.

[0030] The text preprocessing submodule performs segmentation, noise removal, and encoding standardization on text data (including policy documents, notices, meeting minutes, service guides, and law enforcement documents) to obtain a standardized text paragraph sequence. Segmentation is performed according to the chapter structure, clause numbering, and paragraph boundaries of the government documents, ensuring each text paragraph corresponds to an independent semantic unit. Noise removal includes removing redundant whitespace characters, correcting garbled characters, and standardizing punctuation. Encoding standardization converts all text to UTF-8 encoding format.

[0031] The table preprocessing submodule performs field recognition, row and column alignment, and format normalization on tabular data (including item tables, approval workflow records, regulatory ledgers, case handling logs, and performance reports) to obtain structured tabular data. Field recognition determines the semantic meaning of each column by parsing the table header information. Row and column alignment handles irregular table structures such as merged cells and cross-row / column structures, converting them into standard row and column matrices. Format normalization standardizes fields such as date, amount, and number to predefined standard formats.

[0032] The image preprocessing submodule performs layout analysis, text region localization, and identification of seals, signatures, and form elements on image data and scanned documents (including form screenshots, on-site photos, and scanned pages from PDF attachments) to obtain structured image description data. Layout analysis divides the image into text regions, chart regions, seal regions, and signature regions. Text region localization determines the spatial coordinates and reading order of each text block. Optical character recognition (OCR) is performed on the localized text regions to obtain the text content and its positional mapping within the image. Seal and signature regions are labeled with their type and location information.

[0033] The audio and video preprocessing submodule performs speech-to-text transcription, key segmentation, and timestamp alignment on audio data (including hotline recordings and meeting recordings) and video data (including audio and video interviews and law enforcement evidence videos) to obtain a time-stamped transcribed text sequence. Speech-to-text transcription uses automatic speech recognition to convert audio signals into text content. Key segmentation divides continuous audio into independent semantic segments based on pauses, topic changes, and speaker variations. Timestamp alignment binds each transcribed text segment to its start and end times in the original audio and video.

[0034] The process log preprocessing submodule performs status extraction, event serialization, and time chain organization on process log data (including approval workflow records and case logs) to obtain a structured event sequence. Status extraction extracts the status information of each workflow node from the log records, including status identifiers such as acceptance, review, correction, approval, and completion. Event serialization arranges multiple log records for the same matter in chronological order, forming a complete workflow event chain. Time chain organization labels each event node with its absolute time and the interval relative to the previous node.

[0035] The multimodal government data access module encapsulates the standardized text paragraph sequences, structured table data, image structured description data, time-stamped transcribed text sequences, and structured event sequences output by the above sub-modules into a standard intermediate representation. Each standard intermediate representation includes the original data identifier, modality type label, preprocessed content body, and source metadata.

[0036] The government affairs semantic governance module is configured to perform semantic governance on standard intermediate representations and generate government affairs knowledge units. The government semantic governance module receives standard intermediate representations and performs data cleaning, deduplication, anonymization, structured extraction, entity recognition, relationship extraction, time normalization, departmental affiliation labeling, policy level labeling, and version labeling, converting each standard intermediate representation into a searchable government knowledge unit.

[0037] The data cleaning and deduplication submodule performs content integrity checks and deduplication checks on the standard intermediate representations. Content integrity checks determine whether each standard intermediate representation contains valid content, filtering out blank and invalid records. Deduplication checks identify and merge duplicate data by calculating the content similarity between standard intermediate representations, retaining the latest or most complete version.

[0038] The desensitization submodule performs desensitization processing on sensitive information in the standard intermediate representation. Sensitive information includes personal privacy information such as natural persons' names, ID numbers, mobile phone numbers, bank account numbers, and addresses, as well as government information marked as internal or confidential. The desensitization process masks or replaces sensitive fields according to predefined desensitization rules, while preserving the mapping relationship before and after desensitization for restoration in authorized scenarios.

[0039] The structured extraction submodule extracts structured attribute fields from the content body of the standard intermediate representation. For policy document content, it extracts attributes such as issuing authority, document number, publication date, effective date, repeal date, applicable subjects, applicable region, and policy level (national, provincial, municipal, district / county). For service guide content, it extracts attributes such as item name, processing conditions, required materials, processing time limit, fee standard, and processing location. For case record content, it extracts attributes such as applicant type, item number, application time, current status, processing department, and completion time.

[0040] The entity recognition and relation extraction submodule performs named entity recognition and relation extraction on the content body of the standard intermediate representation. Named entity recognition extracts entities from the government domain, including item names, department names, policy file names, material names, processing conditions, time expressions, regional names, personnel roles, and monetary values. Relation extraction identifies semantic relationships between entities, including "item-required materials," "item-processing department," "item-applicable conditions," "policy document-stipulated items," "policy document-repealed policy documents," and "item-processing time limit." The identified entities and relationships are used to construct a government entity relationship network. This network uses items, policy documents, materials, departments, conditions, time limits, and processing results as nodes, and the aforementioned relationship types as edges, forming a relational structure of government knowledge.

[0041] The Time Normalization and Version Labeling submodule normalizes the time expressions in the standard intermediate representation, converting relative time expressions such as "last year," "last month," and "last three years" into absolute time ranges. For multiple versions of the same matter or policy document, version numbers are labeled according to the publication and effective dates, and the currently valid version and historical versions are marked. For repealed policy documents, the repeal status and the basis for repeal are indicated.

[0042] The department affiliation labeling submodule labels the business department to which each standard intermediate representation belongs, based on the source metadata of the standard intermediate representation and the department entity in the content body. For joint documents or collaborative matters involving multiple departments, it labels all relevant departments and their respective responsibilities.

[0043] The aforementioned government semantic governance module converts each standard intermediate representation, after the above processing, into a government knowledge unit. Each government knowledge unit includes the following attributes: data source, business department, policy level, item name, applicable object, time validity (including effective and expiration times), geographical scope, key conditions, evidence location (i.e., the specific location in the original data corresponding to the government knowledge unit, such as the article / paragraph number of a policy document, the specific area coordinates in an image, or the specific time period in an audio recording), related items (determined through a government entity relationship network), and permission level. All government knowledge units and their associated government entity relationship networks together constitute the government knowledge foundation.

[0044] The multimodal semantic encoding module is configured to perform semantic encoding on government knowledge units and generate a unified government semantic space. The multimodal semantic encoding module performs semantic encoding on different modalities of content within the government knowledge unit, generating basic semantic vectors. These basic semantic vectors are then mapped to a unified government semantic space through cross-modal alignment. Before encoding, Z-score standardization is performed on numerical attribute fields (such as amount, time limit, and frequency) in each modality to eliminate the influence of different units on subsequent vector operations. One-hot encoding is performed on categorical attribute fields such as status identifiers and department categories to convert them into numerical representations.

[0045] The modal branch coding submodule assigns government knowledge units to corresponding coding branches based on their modal type labels.

[0046] The text encoding branch uses a pre-trained text encoder to generate text semantic vectors for text-based government knowledge units (including policy provisions, service guides, notices, and announcements in standardized text paragraph sequences). The pre-trained text encoder is a Transformer-based encoder model. The input layer receives the word sequence obtained after word segmentation of the text content of the government knowledge unit, and the output layer outputs fixed-dimensional text semantic vectors. Training is performed on a large-scale government corpus using a masked language model pre-training, and the optimization strategy employs the Adam optimization algorithm.

[0047] The table encoding branch, for structured tabular data-type government knowledge units, linearizes the row and column structure of the structured tabular data into a text sequence with field labels, and then uses a pre-trained text encoder to generate a table semantic vector. The linearization process converts each row of data into the format "field name 1: field value 1; field name 2: field value 2; ...", preserving the table's structural semantics. The linearized text sequence serves as the input to the pre-trained text encoder, outputting a fixed-dimensional table semantic vector.

[0048] The image coding branch processes the textual and visual content of the image-structured descriptive data for government knowledge units. For the textual content, optical character recognition results are used to generate semantic vectors through a pre-trained text encoder, with an output dimension of [missing information]. ,in This represents the dimension of the text semantic vector. The visual content part uses a pre-trained visual encoder to generate visual semantic vectors from the original image regions, with an output dimension of [dimensionality missing]. ,in The dimension of the visual semantic vector is represented by a CNN-based image encoding model. The input layer receives the pixel matrix of the image region, and the output layer outputs a fixed-dimensional visual semantic vector. Training is performed on a large-scale image dataset for image classification pre-training, and the optimization strategy uses the Adam optimization algorithm. The semantic vector of the text and its dimension are After concatenating the visual semantic vectors, we get a dimension of The concatenated vector is then mapped to a dimension through a linear projection layer. The image-comprehension semantic vector, where The vector dimension representing the unified government semantic space. The dimensions are the same as those of the text semantic vector and the table semantic vector to ensure that the dimensions of each modal vector are consistent during subsequent cross-modal alignment.

[0049] The audio encoding branch, for time-stamped transcribed text sequences of government knowledge units, uses a pre-trained text encoder to generate audio-transcribed semantic vectors from the transcribed text content, with an output dimension of [missing information]. It also preserves the correspondence between each audio transcription semantic vector and the original audio time segment.

[0050] The process coding branch, for structured event sequence-type government knowledge units, concatenates the status information, time information, and processing department information of each node in the structured event sequence into a text description, and then uses a pre-trained text encoder to generate a process semantic vector, with an output dimension of [missing information]. .

[0051] The cross-modal alignment submodule maps the different modal semantic vectors output from the aforementioned encoding branches to a unified government semantic space. This submodule is a collection of multiple modality-specific linear projection layers. Each linear projection layer maps the basic semantic vector of its corresponding modality to a unified-dimensional government semantic space vector. Each linear projection layer is trained using a contrastive learning approach: data pairs describing the same government content from different modalities (e.g., a text version of a policy document and its scanned version, a text description of a service guide and its corresponding form screenshot, a transcribed text of a hotline recording and its corresponding processing record) are used as positive sample pairs, while data pairs describing different government content are used as negative sample pairs. The loss function is a contrastive loss.

[0052] in and The first The unified government semantic space vectors obtained by mapping the two modalities in a positive sample pair through their respective linear projection layers have a dimension of 1. , The index for positive sample pairs. For cosine similarity, For temperature coefficient, This refers to the number of samples within a batch. The traversal index of samples within the batch and Take from 1 , The function is denoted as exponential, and the optimization strategy employs the Adam optimization algorithm. By minimizing the aforementioned contrastive loss, the distances between semantically identical but different modal vectors are brought closer together in a unified government semantic space, while the distances between semantically dissimilar vectors are increased.

[0053] After cross-modal alignment, each government knowledge unit obtains a dimension. A unified government semantic vector is generated and stored in a vector index library. Simultaneously, it is associated with the attribute information of government knowledge units and nodes in the government entity relationship network. The unified government semantic vector, the vector index library, the attribute information of government knowledge units, and the government entity relationship network together constitute the unified government semantic space.

[0054] The intent recognition and task routing module is configured to parse user input and assign tasks. The intent recognition and task routing module receives user input, which can be in the form of natural language text, voice, images, or a combination of the above.

[0055] The input modality discrimination submodule determines the modality type of the user input. For voice input, it calls automatic speech recognition to convert the voice input into text. For image input, it calls optical character recognition to extract the text content from the image input, while preserving the image itself for subsequent image retrieval. For mixed input, each modality is processed separately, and the processing results are merged into a unified query text. After processing by the input modality discrimination submodule, the user query text and any accompanying image query content are obtained.

[0056] The intent recognition submodule performs intent classification on user query text. This submodule uses a pre-trained language model to analyze the user query text and identify the business intent category to which the query text belongs. The input layer of the pre-trained language model receives the word sequence obtained after word segmentation of the user query text, and the output layer outputs the probability distribution of each business intent category through a linear classification layer. The category with the highest probability is taken as the recognition result. The training method involves supervised fine-tuning on government query samples labeled with business intent categories. The loss function uses cross-entropy loss, and the optimization strategy uses the Adam optimization algorithm. Business intent categories include: policy retrieval (querying specific policy provisions, application conditions, applicable objects, or time limits), material verification (comparing whether the application materials are complete and meet the requirements), case comparison (finding historical similar cases and their processing results), trend analysis (analyzing complaint hotspots, processing bottlenecks, or regional differences), process tracking (querying the approval status of specific matters), and knowledge Q&A (general consultation on government affairs knowledge or processing procedures).

[0057] The complex query decomposition submodule performs multi-dimensional condition decomposition on the user query text, extracting constraint information from the following dimensions: Subject identification identifies the type of object involved in the user's query text, including individuals, businesses, projects, or matters; Condition recognition extracts the limiting conditions contained in the user's query text, including geographical restrictions, time restrictions, industry restrictions, qualification conditions, amount conditions, and frequency conditions; Target identification determines the type of results users expect to obtain, including policy provisions, material lists, processing paths, case references, anomaly alerts, or statistical analysis. Constraint identification identifies additional constraints in user query text, including whether the latest version is required, whether filtering is done by a specific department, and whether output should be based on permission level.

[0058] The complex query decomposition submodule organizes the above extraction results into a structured query description, which includes intent category, subject information, condition set, target type, and constraint set.

[0059] The task routing submodule generates a task routing plan based on the structured query description. The task routing plan specifies the target data range to be retrieved for the current query, including the policy library (corresponding to policy documents / government knowledge units), case library (corresponding to case handling records / government knowledge units), document library (corresponding to law enforcement documents and notices / announcements / government knowledge units), process library (corresponding to structured event sequences / government knowledge units), image library (corresponding to image structured description data / government knowledge units), and audio library (corresponding to time-stamped transcribed text sequences / government knowledge units). For complex queries involving multiple business intent categories, the task routing plan contains multiple parallel or sequential retrieval subtasks, each specifying its corresponding retrieval channel and retrieval conditions.

[0060] The hybrid retrieval module is configured to perform multi-channel hybrid recall and obtain a set of candidate results. The hybrid retrieval module performs multi-channel hybrid retrieval in a unified government semantic space according to the task routing plan to obtain a set of candidate results.

[0061] The keyword recall submodule extracts keywords from user query text and structured query descriptions, including policy file names, item names, document numbers, serial numbers, and specific terms. It performs exact matching and fuzzy matching on the attribute fields and text content of government knowledge units to obtain a keyword recall result set.

[0062] The vector recall submodule uses the text encoding branch of the multimodal semantic encoding module to generate a dimension of the user query text. The query semantic vector is generated and mapped to the unified government semantic space via the corresponding linear projection layer in the cross-modal alignment submodule. If the user input includes image query content, the image encoding branch is used to generate a dimension of [missing information - likely a specific dimension]. The query image semantic vector is also mapped to the unified government semantic space through a corresponding linear projection layer. The query semantic vector and the query image semantic vector are then mapped to vector indexes of the same dimension. The cosine similarity of the unified government semantic vectors is calculated, and the vectors are sorted in descending order of similarity to obtain the vector recall result set.

[0063] The structured filtering submodule performs structured filtering on the attribute fields of government knowledge units based on the set of conditions and constraints in the structured query description. Filtering conditions include business department matching, geographical scope matching, time validity matching (retaining only currently valid government knowledge units), policy level matching, applicable object matching, and permission level matching (retaining only government knowledge units accessible to the current user with the appropriate permission level). The structured filtering submodule applies the filtering conditions to the keyword recall result set and the vector recall result set, removing result items that do not meet the conditions to obtain the filtered recall result set.

[0064] The graph association submodule starts with the government knowledge units in the filtered recall result set and performs extended searches along the relationship edges in the government entity relationship network. The extended search expands outwards along relationship types such as "Item—Required Materials," "Item—Handling Department," "Item—Applicable Conditions," "Policy Document—Regulations," and "Policy Document—Repealed Policy Documents" to obtain government knowledge units directly associated with the filtered recall result set, serving as a supplementary result set for the graph association. The depth of the extended search is limited to two hops, meaning it can be reached from the starting node through a maximum of two relationship edges.

[0065] The timeliness filtering submodule performs timeliness verification on the filtered recall result set and the graph association supplementary result set. Timeliness verification determines whether a government knowledge unit is within its validity period based on its time validity attribute. For the same policy document or service guide with multiple versions, the latest valid version is prioritized, while obsolete and expired versions are reduced in sorting weight but not completely removed (to support historical version comparison needs).

[0066] The hybrid retrieval module combines the filtered recall result set and the map association supplementary result set, and removes duplicates to obtain the candidate result set.

[0067] The evidence chain analysis module is configured to perform correlation sorting and evidence chain organization on the candidate result set. The evidence chain analysis module performs association sorting, source verification, version comparison, context completion, and evidence chain organization on the government knowledge units in the candidate result set to obtain a structured evidence chain.

[0068] The correlation ranking submodule calculates a comprehensive relevance score for each government affairs knowledge unit in the candidate result set relative to the user query. All scores involved in the weighted summation are normalized before calculation. Intervals were used to eliminate dimensional differences between scores from different sources. The overall relevance score was then considered. It is obtained by weighted summation of the following five factors:

[0069] in To achieve a comprehensive relevance score, These are the weight parameters corresponding to each factor. To calculate the vector similarity score, the cosine similarity value between the query semantic vector and the unified government semantic vector of the government knowledge unit is taken and normalized to... interval; The keyword hit score is calculated by taking the percentage of the query keyword that hits the attribute field and text content of the government knowledge unit. The value range is [value range missing]. ; To calculate the graph relevance score, for each government knowledge unit obtained through graph relevance supplementation, the score is calculated based on the number of extended hops between the government knowledge unit and the recall starting node. reciprocal ,in This is the number of hops between the government knowledge unit and the recall starting node. For directly recalled government knowledge units, this value is 1, and the range is [value missing]. ; For timeliness scoring, the current valid version is assigned a value of 1, and historical versions are assigned values ​​decreasing sequentially from newest to oldest, with a value range of [value missing]. ; The permission matching score is 1 if the permission level of the government knowledge unit exactly matches the current user's data access level, and 0 otherwise. The score is based on the comprehensive relevance score. Sort the candidate result set in descending order. Weight parameters The initial value is obtained by performing supervised training on a government affairs query sample set labeled with relevance tags, and is continuously updated by the feedback write-back submodule in the access control and audit module based on user evaluation feedback.

[0070] The source verification submodule verifies the validity of the evidence location for each government knowledge unit in the sorted candidate result set. Verification operations include: verifying the existence of the policy provision number cited by the government knowledge unit in the original policy document; verifying whether the coordinates of the cited image area correspond to valid content in the original image; verifying whether the cited audio time segment contains valid transcription content in the original audio; and verifying whether the cited case record number has a corresponding entry in the process log. Government knowledge units that fail verification are marked as "Source Pending Confirmation".

[0071] The version comparison submodule compares multiple versions of the same policy document or service guide within the candidate result set. Version comparison extracts differences between different versions in key fields such as applicable conditions, required documents, time limits, and fee standards, generating version difference records. These records annotate the changed fields, the content before the change, the content after the change, and the effective date of the change.

[0072] The context completion submodule performs context completion on incomplete government knowledge units within the sorted candidate result set. Context completion extracts the content of the preceding and following paragraphs from the original data based on the evidence location of the government knowledge unit, appending this context information to the unit. For image-based government knowledge units, context completion extracts content from other areas on the same page. For process-based government knowledge units, context completion extracts the preceding and following flow nodes for the same matter.

[0073] The evidence chain organization submodule organizes the candidate result set after the above processing according to the structure of "Conclusion-Basis-Source-Time-Version-Scope of Application". Each possible conclusion is bound to at least one source evidence, and the source evidence is marked with its specific source (policy document name and clause number, case record number, image file name and regional coordinates, audio file name and time period). When multiple pieces of evidence support the same conclusion, cross-verification is performed, and the consistency status between the evidence is marked. When there are conflicting pieces of evidence (such as different versions of policy documents stipulating different handling conditions for the same matter), they are sorted according to version priority, with the latest valid version of the evidence listed first. For government knowledge units with a comprehensive relevance score lower than a preset threshold, they are marked as "low confidence" and a secondary search suggestion or manual review prompt is generated. The evidence chain organization submodule outputs a structured evidence chain. Each record in the structured evidence chain includes a conclusion summary, evidence content, source identifier, time stamp, version information, scope of application, and confidence level.

[0074] The large model generation and verification module is configured to generate government responses based on structured evidence chains and perform verification. The large model generation and verification module takes a structured evidence chain as input, calls a large language model to generate government response text, and performs multi-layer verification on the generated results.

[0075] The prompt assembly submodule assembles the conclusion summary, evidence content, source identifiers, and scope of application from the structured evidence chain into input prompts for the large language model. The input prompts include the following constraints: generate responses only based on the provided evidence content; label the source file name and clause number for cited policy provisions; assign confidence levels to uncertain content; and prohibit the generation of factual information not included in the evidence.

[0076] The response generation submodule inputs the assembled input prompts into the large language model to obtain a draft of the government response. The content type of the draft government response is determined according to the user's query target type, including policy interpretation, processing procedure description, list of required materials, case reference analysis, abnormal risk warning or statistical analysis report.

[0077] The rule validation submodule performs rule validation on the initial draft of government response, including: The key information is strongly verified. The policy article number, amount, time limit and processing conditions appearing in the draft of the government response are extracted and compared with the original content of the corresponding evidence in the structured evidence chain. Inconsistent content is marked and corrected. Source verification involves checking whether each citation in the initial draft of the government response has corresponding source evidence in the structured chain of evidence, and deleting or marking citations whose sources cannot be traced as "to be verified"; Consistency verification checks whether there are contradictory statements in the initial draft of government response (such as different processing conditions described in different paragraphs for the same matter), and unifies contradictory statements according to the version priority in the structured evidence chain.

[0078] The confidence level labeling submodule labels the confidence level of content paragraphs generated based on "low confidence" evidence in the initial draft of government responses, prompting users that the reliability of this part of the content is low and suggesting further verification.

[0079] The de-identification display submodule performs de-identification display on sensitive information in the initial draft of government responses based on the current user's permission level and the permission level attributes of the government knowledge unit. The scope and granularity of the de-identification display are determined by the dynamic de-identification rules provided by the access control and auditing module.

[0080] The format standardization submodule performs format standardization on the government response after verification and anonymization, generating the final government response. The format standardization selects the corresponding output template according to the output scenario (service window display, back-end review report, or leader decision summary), and typeset the response content, evidence citations, source annotations, version information, and confidence level annotations according to the output template format.

[0081] The access control and auditing module is configured to perform end-to-end access control and behavior logging. The access control and auditing module runs through the entire process of the large-scale multimodal government data intelligent retrieval and analysis system. It performs access control in the retrieval, generation and output stages and records all operations.

[0082] The identity authentication and permission resolution submodule retrieves the user's identity information when a user initiates a query, including user ID, business role, department, administrative region, and data access level. Based on predefined permission policies, it resolves the range of data the current user can access, including the range of accessible business departments, policy levels, permission levels, and geographical areas. The permission resolution results are then passed to the structured filtering submodule of the hybrid search module to filter out government knowledge units that the user is not authorized to access during the search phase.

[0083] The dynamic data masking rules submodule generates dynamic data masking rules based on the user's data access level and the permission level of the government knowledge unit. These rules specify the appropriate masking operation (complete masking, partial masking, or summary replacement) to be performed on sensitive fields in government knowledge units that exceed the user's access level. The dynamic data masking rules are then passed to the data masking display submodule of the large model generation and validation module.

[0084] The behavior recording submodule records the following operational behaviors across the entire process: user query behavior (including query time, query content, query modality, and user identity), retrieval hit behavior (including the identifier of the hit government knowledge unit, the type of recall channel, and the comprehensive relevance score), evidence citation behavior (including the identifier and source information of the evidence cited in the final government response), result export behavior (including export time, scope of exported content, and export format), and manual correction behavior (including correction time, correction content, and corrector identifier). All behavior records are stored in the audit log database for subsequent compliance audits and operational traceability.

[0085] The feedback write-back submodule receives user feedback on corrections and evaluations of the final government response. For user corrections (e.g., a user pointing out an updated policy provision or changed material requirements), the correction information is written back to the government semantic governance module, triggering an update verification of the corresponding government knowledge unit. For user evaluations (e.g., a user indicating the response is accurate or inaccurate), the evaluation information is written back to the hybrid retrieval module and the evidence chain analysis module to adjust the weighting parameters of each factor in the comprehensive relevance score. We will continue to optimize recall accuracy and sorting precision.

[0086] Technical effects of this embodiment This implementation method uses a multimodal government data access module to perform targeted preprocessing on government data of different modalities, such as text, tables, images, scanned documents, audio and video, and process logs, and encapsulates them into a unified standard intermediate representation. This makes subsequent processing no longer limited to a single modality, thereby overcoming the problem that traditional government retrieval systems cannot uniformly understand multimodal government data due to the fragmentation of data types.

[0087] This implementation uses a cross-modal alignment submodule within the multimodal semantic coding module to map basic semantic vectors from different modalities to a unified government semantic space using a contrastive learning approach. This allows queries submitted by users in the form of images, voice, or spoken language to perform similarity calculations with government knowledge units from different modalities, such as text-based policy articles and form-based application records, within the unified government semantic space. This overcomes the problem of cross-modal retrieval being impossible due to the lack of a unified semantic mapping between different modal data.

[0088] This implementation method uses the complex query decomposition submodule in the intent recognition and task routing module to break down user query text into structured query descriptions with multiple dimensions such as subject, conditions, target, and constraints. The task routing submodule then assigns the structured query descriptions to the corresponding retrieval channels. This enables the large-scale multimodal government data intelligent retrieval and analysis system to handle colloquial and fuzzy queries with multiple constraints, thereby overcoming the problem that traditional systems rely solely on keyword matching and cannot accurately identify complex government query intents.

[0089] This implementation method integrates five retrieval methods—keyword retrieval, vector retrieval, structured filtering, graph association, and timeliness filtering—through a hybrid retrieval module. This allows for complementarity between exact matching and semantic matching, between structured constraints and unstructured semantics, and between direct and indirect associations. As a result, it overcomes the problem of incomplete or insufficient retrieval accuracy of a single retrieval method in complex government affairs queries.

[0090] This implementation method binds each conclusion to specific source evidence through the evidence chain analysis module and marks the source identifier, time stamp, version information and scope of application. By cross-verification and version priority sorting of conflicting evidence, the integrity and consistency of the structured evidence chain are guaranteed, thereby overcoming the problem that existing systems only output conclusions without providing evidence chains, resulting in untraceable results.

[0091] This implementation method uses the time normalization and version labeling submodule in the government semantic governance module and the timeliness filtering submodule in the hybrid retrieval module to perform version labeling and timeliness verification on government knowledge units, prioritizing the retention of the latest valid versions and marking them as obsolete. This overcomes the problem that frequent policy updates cause the large-model multimodal government data intelligent retrieval and analysis system to output outdated content.

[0092] This implementation method uses an access control and auditing module to perform access filtering during the retrieval stage, dynamic desensitization during the generation stage, and behavior recording during the output stage, forming a full-link control system that includes filtering before retrieval, constraints during generation, and traceability after output. This overcomes the problem that traditional intelligent retrieval systems lack granularity in access filtering, desensitization display, and auditing traceability, thus failing to meet the requirements of government security compliance.

[0093] The Municipal Administrative Approval Bureau is responsible for accepting applications for construction permits for building projects within its jurisdiction. The bureau's government database stores relevant policy documents (including historical versions), service guides, approval process records, site inspection photos, and hotline recordings. One day, a staff member from the bureau's business department (user ID: USR-0471, business role: department auditor, data access level: level three) submitted a query through the system: "We have a company applying for a construction permit with an investment of 38 million yuan. We would like to confirm what materials are currently required and whether there have been any recent changes." This query, entered in natural language text format, triggered the system's entire processing flow.

[0094] Multimodal government data access module Before receiving the query, the system had already preprocessed the data related to construction permits in the government database and encapsulated it into a standard intermediate representation. The raw data involved in this search covers three modalities: text-based policy documents, table-based approval workflow records, and image-based scanned documents. The processing procedures for each sub-module are as follows.

[0095] The text preprocessing submodule segments the "Administrative Measures for Construction Permits of a Certain City" according to its chapter structure, identifying "Article 12. To apply for a construction permit, the construction unit shall provide the following materials..." as an independent semantic unit. After noise removal and UTF-8 encoding standardization, it outputs a standardized text paragraph sequence. The table preprocessing submodule performs field recognition and format standardization on the approval flow record table, standardizing the date field to "YYYY-MM-DD" format and the amount field to an integer format with "yuan" as the unit, outputting structured table data. The image preprocessing submodule performs layout analysis on the site survey photos and scanned copies of the service guide, locates the text areas, performs optical character recognition, and outputs structured image description data.

[0096] Table 1. Examples of Standard Intermediate Representations

[0097] Government Semantic Governance Module The government semantic governance module performs structured extraction, entity recognition, relation extraction, and version labeling on the above-mentioned standard intermediate representations, converts them into government knowledge units, and constructs a government entity relationship network.

[0098] The structured extraction submodule extracts attributes such as issuing authority, document number, publication date, effective date, repeal date, and applicable objects from DOC-2201 and DOC-2198. The time normalization and version labeling submodule identifies that DOC-2201 (Year A version) has been repealed by DOC-2198 (Year B version), labels DOC-2201 as a historical version (version number 1), labels DOC-2198 as the currently valid version (version number 2), and records the repeal basis as "Municipal Government Order No. 18 of Year B". The entity identification and relationship extraction submodule extracts material name entities such as "Construction Permit", "Construction Project Planning Permit", "Construction Land Planning Permit", and "Engineering Quality and Safety Supervision Procedures" from DOC-2198, and establishes a "Construction Permit Application Item - Required Materials" relationship edge, incorporating it into the government entity relationship network.

[0099] Table 2 Examples of attributes for government knowledge units

[0100] In the government entity relationship network, a "repeal" relationship edge is established between KU-3301 and KU-3298, and a "item-required material" relationship edge is established between KU-3301 and material knowledge units such as "construction project planning permit" and "engineering quality and safety supervision procedures", forming a relational structure with the construction permit application item as the central node.

[0101] Multimodal semantic coding module The multimodal semantic coding module performs submodal coding and cross-modal alignment on the aforementioned government knowledge units.

[0102] The modal branch encoding submodule assigns KU-3301 and KU-3298 to the text encoding branch, and uses a pre-trained text encoder to generate a dimension of... The text semantic vector is generated. KU-3412 (table type) is assigned to the table encoding branch. First, linearization is performed, converting each line of the approval process record into a text sequence in the format "Node Name: Acceptance; Processing Department: Comprehensive Acceptance Window; Processing Time: Year C, January 8th; Status: Completed". Then, a pre-trained text encoder is used to generate a dimension of... The table semantic vector. IMG-1042 (image class) is assigned to the image coding branch, and the text content is generated by a pre-trained text encoder with dimension [missing information]. The text semantic vectors are generated, and the visual content is generated by a pre-trained visual encoder with dimensions of [missing information]. The visual semantic vectors are concatenated to obtain a dimension of [dimensional value]. The concatenated vectors are then mapped to the dimension via a linear projection layer. The image-based semantic vector.

[0103] The cross-modal alignment submodule maps the basic semantic vectors of each modality to a unified government semantic space through their respective linear projection layers. This is based on the text semantic vectors of KU-3301. Image-to-semantic vectors of IMG-1042 Positive sample pairs (both describing the contents of the construction permit material list) are formed, and the vectors of the remaining government knowledge units within the batch are used as negative samples. The comparative loss is calculated as follows:

[0104] Temperature coefficient Batch size By minimizing The cosine similarity between KU-3301 and IMG-1042 in the unified government affairs semantic space is made close to 1, while the cosine similarity of semantically unrelated government affairs knowledge units (such as tax registration knowledge units) is made close to 0. After alignment, each government affairs knowledge unit obtains a unified government affairs semantic vector with a dimension of 768 and stores it in the vector index library.

[0105] Intent recognition and task routing module The user's query text, "We have a company applying for a construction permit with an investment of 38 million yuan. We would like to confirm what materials are needed and if there are any recent changes," is confirmed as plain text input by the input modality discrimination submodule and directly proceeds to the intent recognition submodule.

[0106] The intent recognition submodule outputs the probability distribution of each business intent category. The "material verification" category has the highest probability (0.71), while the "policy retrieval" category has the second highest probability (0.22). The system recognizes that the query involves both material verification and policy retrieval intents.

[0107] The complex query decomposition submodule performs multi-dimensional condition decomposition on the user query text and extracts the structured query description as follows: Table 3. Description of Structured Queries

[0108] Based on this, the task routing submodule generates a task routing plan: it launches the policy library retrieval subtask (target KU-3301 and its version history) and the case library retrieval subtask (target historical case records of similar investment amounts) in parallel, both of which specify vector recall and structured filtering to be executed together.

[0109] Hybrid search module The hybrid retrieval module performs multi-channel recall based on the task routing plan.

[0110] The keyword recall submodule extracts keywords "construction permit," "materials," "38 million," and "latest" from the structured query description. It then performs exact matching on the attribute fields of the government knowledge unit, matching KU-3301, KU-3298, and KU-3412, forming a keyword recall result set. The vector recall submodule generates a 768-dimensional query semantic vector from the user's query text. After mapping it to the unified government semantic space through a corresponding linear projection layer, it calculates the cosine similarity with all unified government semantic vectors in the vector index library. The top three similarity results are as follows: Table 4. Vector Recall Similarity Results

[0111] The structured filtering submodule performs time validity matching based on the constraint set. KU-3301 passes the filter (currently valid), KU-3298 is retained but downgraded (historical version), and KU-3412 passes the filter (accessible at user data access level 3). The graph association submodule starts with KU-3301 and extends outward one hop along the "item-required materials" relationship edge to supplement the graph association results for "Construction Project Planning Permit Material Description (KU-2801)" and "Engineering Quality and Safety Supervision Procedure Description (KU-2803)". The timeliness filtering submodule confirms KU-3301 as the current valid version and marks KU-3298 as a historical version for retention to support version comparison. After merging and deduplicating, the candidate result set contains a total of 5 government knowledge units.

[0112] Evidence Chain Analysis Module The correlation ranking submodule calculates a comprehensive relevance score for each government knowledge unit in the candidate result set. ,in , For vector semantic similarity, Scoring is based on keyword hits. For the graph association score, Score based on timeliness. The weight parameter is taken as the current value for the permission matching score. For example, KU-3301:

[0113] Table 5. Ranking of candidate result sets by comprehensive relevance score

[0114] The source verification submodule verifies that "Article 12, Paragraph 1 of DOC-2198" cited in KU-3301 contains valid content in the original policy document, and the verification passes. The version comparison submodule performs a field-level comparison between KU-3301 and KU-3298, identifying that the B version adds a material requirement of "copy of construction project safety production license" and deletes the requirement of "proof of construction funds in place" in the A version, with the effective date being December 1, B year. The context completion submodule extracts Article 11 (application conditions) and Article 13 (time limit provisions) from DOC-2198 as context information for KU-3301. The evidence chain organization submodule organizes the above processing results into a structured evidence chain according to the structure of "Conclusion-Basis-Source-Time-Version-Scope of Application", forming a total of 3 records: the current list of valid materials (bound to KU-3301, source identifier DOC-2198 Article 12, Paragraph 1), version change differences (bound to KU-3298 compared with KU-3301, 2 changed fields), and related material descriptions (bound to KU-2801 and KU-2803).

[0115] Large model generation and verification module The prompt assembly submodule assembles the conclusion summary of KU-3301 ("List of materials required for construction permit application in Year B, 8 items in total"), evidence content (full text of Article 12, Paragraph 1 of DOC-2198), source identification, and version change difference information from the structured evidence chain into input prompts, with the additional constraint instruction: "Generate a response based only on the provided evidence content; mark the source file name and clause number of the cited policy provisions; prohibit the generation of factual information not included in the evidence."

[0116] The response generation submodule outputs a draft of the government response, with the content type being "list of materials" and "version change explanation". The rule verification submodule compares each of the "8 materials" appearing in the draft with the original content of the structured evidence chain, confirming consistency; source verification confirms that each citation can be traced back to DOC-2198; consistency verification finds no contradictory statements. The confidence level labeling submodule confirms that all evidence is of high confidence and requires no additional labeling. The anonymization display submodule confirms that the KU-3301 access level is level two based on the level three data access level of USR-0471, and no anonymization processing is required. The format standardization submodule outputs a template formatted according to the "department review report" to generate the final government response, the main content of which is excerpted below: According to the "Administrative Measures for Construction Permits of a Certain City" (Year B, DOC-2198, Article 12, Paragraph 1), the current list of valid materials consists of 8 items, with 2 changes compared to Year A: "Copy of Construction Safety Production License" has been added, and "Proof of Construction Funds in Place" has been deleted. The above changes will take effect on December 1, Year B.

[0117] Access Control and Auditing Module The identity authentication and permission resolution submodule parses the permission scope of USR-0471 when a user initiates a query: Accessible business departments are all departments of the Administrative Approval Bureau; policy levels are at the municipal level and below; access permission levels are level three and below (including level two); and the accessible geographical area is the entire city. The permission resolution result is passed to the structured filtering submodule of the hybrid search module, which filters out government knowledge units with permission levels higher than level three during the search phase. The dynamic desensitization rule submodule determines that all knowledge units involved in this search are within the user's permission scope and generates a "no desensitization required" rule, which is then passed to the desensitization display submodule. The behavior recording submodule records the entire query process: Table 6 Audit Log Records

[0118] Throughout the entire process, data originates from original government documents (DOC-2201, DOC-2198) and approval flow records (TAB-0583). It is then encapsulated into a standard intermediate representation by the multimodal government data access module, converted into government knowledge units carrying version status, evidence location, and relationships by the government semantic governance module, mapped to a unified government semantic space by the multimodal semantic encoding module, and after intent parsing and task allocation by the intent recognition and task routing module, the hybrid retrieval module jointly recalls candidate result sets through keywords, vectors, and graphs. After the evidence chain analysis module completes sorting, version comparison, and evidence chain organization, the large model generation and verification module generates the final government response based on the structured evidence chain and verifies it according to rules. The access control and auditing module ensures compliance with access control throughout the process and fully records operational behavior. The intermediate data output by each module (standard intermediate representation → government knowledge unit → unified government semantic vector → candidate result set → structured evidence chain → final government response) remains consistent at the semantic level. Version change information (two differences between version A and version B) runs from the version label of the government semantic governance module to the change description of the final government response, reflecting the complete closed loop of data flow.

[0119] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A large-scale, multimodal intelligent retrieval and analysis system for smart government data, characterized in that: include: The multimodal government data access module is configured to access raw government data in multiple modalities, perform preprocessing for different modalities, and encapsulate the preprocessing results into a standard intermediate representation. The government semantic governance module is configured to perform data cleaning, deduplication, desensitization, structured extraction, entity recognition, relationship extraction, time normalization, and version labeling on the standard intermediate representation, generating government knowledge units and government entity relationship networks; The multimodal semantic encoding module is configured to generate basic semantic vectors for different modal contents in government knowledge units through corresponding encoding branches, and to map the basic semantic vectors of each modality to a unified government semantic space through cross-modal alignment, thereby generating a unified government semantic vector and storing it in a vector index library. The intent recognition and task routing module is configured to perform modality discrimination and intent classification on user input, decompose user queries into structured query descriptions containing intent categories, subject information, condition sets, target types, and constraint sets, and generate task routing plans. The hybrid retrieval module is configured to perform multi-channel hybrid retrieval based on the task routing plan, integrating keyword recall, vector recall, structured filtering, graph association, and time-sensitive filtering to obtain a set of candidate results. The evidence chain analysis module is configured to perform association sorting, source verification, version comparison and context completion on the candidate result set, bind each conclusion to at least one source evidence, and output a structured evidence chain; The large model generation and verification module is configured to generate government responses based on a structured evidence chain by calling a large language model, and to perform strong verification of key information, source verification, and consistency verification on the generated results. The access control and auditing module is configured to perform access filtering during the retrieval phase, provide dynamic de-identification rules during the generation phase, and perform behavior logging during the output phase.

2. The intelligent retrieval and analysis system for large-scale multimodal government data based on smart government as described in claim 1, characterized in that: The multimodal government data access module includes: The text preprocessing submodule performs segmentation, noise removal, and encoding standardization on text data according to chapter structure and clause number to obtain a standardized text paragraph sequence. The table preprocessing submodule performs field recognition, row and column alignment, and format normalization on tabular data to obtain structured table data; The image preprocessing submodule performs layout analysis, text region localization, and optical character recognition on image data and scanned documents to obtain structured image description data; The audio and video preprocessing submodule performs speech transcription, key segmentation, and timestamp alignment on audio and video data to obtain a time-stamped transcribed text sequence. The process log preprocessing submodule performs state extraction, event serialization, and time chain organization on process log data to obtain a structured event sequence. Each standard indicates that it contains the original data identifier, modality type label, preprocessed content body, and source metadata.

3. The intelligent retrieval and analysis system for large-scale multimodal government data based on smart government affairs as described in claim 1, characterized in that: In the aforementioned government semantic governance module: The entity recognition and relation extraction submodule performs named entity recognition and relation extraction on the content of the standard intermediate representation. Entity types include item name, department name, policy file name, material name, processing conditions, time expression, region name, personnel role, and monetary value. Relationship types include semantic relationships between item and required materials, item and processing department, item and applicable conditions, policy document and stipulated item, and policy document and repealed policy document. The government entity relationship network uses items, policy documents, materials, departments, conditions, time limits, and processing results as nodes, and relationship types as edges; The Time Normalization and Version Labeling submodule converts relative time expressions into absolute time intervals, labels multiple versions of the same matter or the same policy document with version numbers, and marks the current valid version, historical versions, and obsolete status. Each government knowledge unit includes attributes such as data source, business department, policy level, item name, applicable objects, time validity, geographical scope, key conditions, evidence location, related items, and authority level.

4. The intelligent retrieval and analysis system for large-scale multimodal government data based on smart government affairs as described in claim 1, characterized in that: In the multimodal semantic coding module: The text encoding branch uses a pre-trained text encoder to generate text semantic vectors for text-based government knowledge units; The table encoding branch linearizes the structured table data into a text sequence with field labels, and then uses a pre-trained text encoder to generate table semantic vectors. The image encoding branch generates a dimension of 1000 from the optical character recognition results using a pre-trained text encoder. The text semantic vectors are generated by a pre-trained visual encoder from the original image region, using a dimension of [missing information]. The visual semantic vectors are concatenated and then mapped to a dimension through a linear projection layer. The image-comprehension semantic vector, where The dimension of the text semantic vector. The dimension of the visual semantic vector. To unify the vector dimension of the government affairs semantic space; The audio encoding branch uses a pre-trained text encoder to generate dimensional data from the transcribed text content. Audio transcription semantic vectors; The process encoding branch concatenates the status, time, and processing department information of each node in the structured event sequence into a text description, and then uses a pre-trained text encoder to generate a dimension of... The process semantic vector.

5. The large-scale multimodal government data intelligent retrieval and analysis system for smart government affairs according to claim 4, characterized in that, The cross-modal alignment is achieved through multiple modality-specific linear projection layers. Each linear projection layer is trained using a contrastive learning approach, and the loss function is: ; in and The first The unified government semantic space vector is obtained by mapping the two modalities in a positive sample pair through their respective linear projection layers. The index for positive sample pairs. For cosine similarity, For temperature coefficient, This refers to the number of samples within a batch. This serves as the traversal index for samples within a batch. This represents an exponential function; positive sample pairs are data pairs describing different modalities of the same government affairs content, while negative sample pairs are data pairs describing different government affairs content.

6. The intelligent retrieval and analysis system for large-scale multimodal government data in smart government affairs according to claim 1, characterized in that, In the intent recognition and task routing module: The intent recognition submodule uses a pre-trained language model to classify user query texts by intent. Business intent categories include policy retrieval, material verification, case comparison, trend analysis, process tracking, and knowledge question answering. The pre-trained language model is subjected to supervised fine-tuning on government query samples labeled with business intent categories. The complex query decomposition submodule extracts subject information, condition information, target information, and constraint information from the user's query text. Subject information includes individuals, enterprises, projects, or matters; condition information includes geographical restrictions, time restrictions, industry restrictions, qualification requirements, monetary requirements, and frequency requirements; target information includes policy provisions, material lists, processing paths, case references, anomaly alerts, or statistical analysis; and constraint information includes version requirements, departmental filtering, and permission level output requirements. The task routing submodule specifies the target data range and retrieval channel based on the structured query description, and generates a task routing plan containing multiple retrieval subtasks for complex queries involving multiple business intent categories.

7. The intelligent retrieval and analysis system for large-scale multimodal government data based on smart government affairs according to claim 1, characterized in that, In the chain of evidence analysis module: The correlation ranking submodule calculates a comprehensive relevance score for each government knowledge unit in the candidate result set. : ; in For weight parameters, For vector similarity scores, Scoring is based on keyword hits. To score the correlation between the graphs, Score based on timeliness. For permission matching scores, all scores are normalized to a certain value before calculation. Interval; the graph correlation score takes the number of extended jumps for government knowledge units obtained through graph correlation supplementation. reciprocal For directly recalled government knowledge units, the value is 1; The weight parameters are obtained by performing supervised training on a sample set of government query data labeled with relevance tags, and are continuously updated based on user feedback. The source verification submodule verifies the validity of the evidence location of each government knowledge unit in the sorted candidate result set one by one, and marks the government knowledge units that fail the verification as having a source pending confirmation. The version comparison submodule extracts the differences between multiple versions of the same policy document or the same service guide, and generates a version difference record that includes the changed fields, the content before the change, the content after the change, and the effective time of the change. The evidence chain organization submodule organizes structured evidence chains according to the structure of conclusion, basis, source, time, version, and scope of application. It performs cross-verification when multiple pieces of evidence support the same conclusion and sorts conflicting evidence according to version priority.

8. The intelligent retrieval and analysis system for large-scale multimodal government data based on smart government as described in claim 1, characterized in that, In the large model generation and verification module: The prompt word assembly submodule assembles the conclusion summary, evidence content, source identifier, and scope of application in the structured evidence chain into input prompt words for the large language model. The input prompt words include constraints such as generating responses based solely on the provided evidence content, marking the source file name and clause number of the cited policy provisions, marking the confidence level of uncertain content, and prohibiting the generation of factual information not included in the evidence. The rule verification submodule performs key information strong verification, source verification, and consistency verification on the draft of government response. The key information strong verification compares the policy article number, amount value, time limit value, and processing conditions in the draft of government response with the original content of the corresponding evidence in the structured evidence chain item by item. The source verification checks whether each citation has a corresponding source evidence in the structured evidence chain. The consistency verification checks for contradictory statements and unifies them according to version priority. The confidence level labeling submodule labels the confidence level of content paragraphs generated based on low-confidence evidence.

9. The intelligent retrieval and analysis system for large-scale multimodal government data based on smart government as described in claim 1, characterized in that, In the access control and auditing module: The identity authentication and permission parsing submodule obtains the user's identity information and parses the data range that the current user can access. The data range includes the range of accessible business departments, policy levels, permission levels, and geographical areas. The permission parsing results are passed to the structured filtering submodule of the hybrid retrieval module. The dynamic desensitization rule submodule generates dynamic desensitization rules based on the user's data access level and the permission level of the government knowledge unit. The dynamic desensitization rules are then passed to the desensitization display submodule of the large model generation and verification module. The behavior logging submodule performs full-link recording of user query behavior, retrieval hit behavior, evidence citation behavior, result export behavior, and manual correction behavior, and stores the records in the audit log library. The feedback write-back submodule receives user correction feedback and writes it back to the government semantic governance module to trigger the update verification of the corresponding government knowledge unit. It also receives user evaluation feedback and writes it back to the evidence chain analysis module to adjust the weight parameters in the comprehensive relevance score.

10. A method for intelligent retrieval and analysis of large-scale multimodal government data for smart government, applicable to the intelligent retrieval and analysis system for large-scale multimodal government data for smart government as described in any one of claims 1 to 9, characterized in that, include: Step 1: Access raw government data of various modalities through the multimodal government data access module, perform preprocessing for different modalities, and encapsulate the preprocessing results into a standard intermediate representation. Step 2: Through the government semantic governance module, perform data cleaning, deduplication, anonymization, structured extraction, entity recognition, relationship extraction, time normalization and version labeling on the standard intermediate representation to generate government knowledge units and government entity relationship networks; Step 3: The multimodal semantic coding module generates basic semantic vectors for different modal contents in the government knowledge unit through corresponding coding branches, and maps the basic semantic vectors of each modality to a unified government semantic space through cross-modal alignment, generating a unified government semantic vector and storing it in the vector index library; Step 4: When a user initiates a query, the intent recognition and task routing module performs modality discrimination and intent classification on the user input, decomposes the user query into a structured query description, and generates a task routing plan. Step 5: Based on the task routing plan, the hybrid retrieval module performs multi-channel hybrid retrieval by integrating keyword recall, vector recall, structured filtering, graph association, and time-sensitive filtering to obtain a set of candidate results. Step 6: Perform association sorting, source verification, version comparison and context completion on the candidate result set through the evidence chain analysis module, bind each conclusion to at least one source evidence, and output a structured evidence chain; Step 7: The large model generation and verification module generates government response based on the structured evidence chain by calling the large language model, and performs strong verification of key information, source verification and consistency verification on the generated result; Step 8: The access control and auditing module performs access filtering, dynamic desensitization, and behavior logging in steps 5 to 7 respectively, and triggers updates to government knowledge units and adjustments to search and sorting parameters based on user feedback.