Government affair event intelligent distribution and processing method and system based on multi-modal large model

Abstract

The invention discloses an intelligent government affair event allocating and processing method and system based on a multi-modal large model, belongs to the technical field of artificial intelligence, and aims to solve the technical problem of how to overcome the defects of much manual intervention, low efficiency and difficulty in multi-modal input analysis in the prior art. According to the technical scheme, intelligent identification, automatic classification and accurate distribution of citizen problems are achieved, the intelligent level and the convenience level of government affair services are improved, and the method comprises the steps that multi-modal data collection and preprocessing are conducted, specifically, modal information of characters, voices and pictures submitted by a government affair client side is received and analyzed, an event initial corpus set is constructed, data cleaning and structural processing are completed, and the data cleaning and structural processing are completed; obtaining the preprocessed multi-modal data; carrying out multi-modal semantic fusion and intention recognition; event classification is matched with responsible departments; performing GAT path reasoning; intelligent work order generation and cross-department collaboration are realized; and performing user feedback and closed-loop optimization.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for intelligent allocation and processing of government affairs events based on a multimodal large model. Background Technology

[0002] Traditional government service handling primarily relies on manual reception of citizens' offline inquiries and telephone hotlines. With the continuous digital upgrades of government service platforms, this is gradually transitioning to online channels such as apps, WeChat mini-programs, and government websites. Citizens' feedback, inquiries, and service requests involve multiple modalities of information, including text, voice, and images, exhibiting characteristics such as complex information structures, diverse event types, and detailed departmental divisions. However, existing government service platforms still heavily depend on human agents to process unstructured information submitted by citizens, facing the following problems:

[0003] ① Manual content parsing is inefficient and inaccurate, especially when dealing with complex, multimodal content.

[0004] ② The event allocation relies on manual rules, making it difficult to intelligently match responsible departments according to citizens' wishes, resulting in misallocation and omissions.

[0005] In existing technologies, some systems have introduced natural language processing (NLP) technology to perform preliminary classification of problems through text analysis. However, for complex events that include multimodal inputs such as images and voice, existing systems have difficulty effectively parsing and processing them.

[0006] In recent years, large language models (such as DeepSeek) have demonstrated outstanding performance in natural language understanding and multimodal content parsing, possessing powerful contextual modeling and reasoning capabilities. Currently, no system has deeply integrated large multimodal models with government event management to achieve automated parsing, classification, and cross-departmental collaborative allocation of multimodal inputs.

[0007] Therefore, how to overcome the shortcomings of existing technologies, such as excessive manual intervention, low efficiency, and difficulty in parsing multimodal inputs, and achieve intelligent identification, automatic classification, and accurate allocation of citizens' problems, thereby improving the intelligence and convenience of government services, is a technical problem that urgently needs to be solved. Summary of the Invention

[0008] The technical objective of this invention is to provide a method and system for intelligent allocation and processing of government affairs based on a multimodal large model, in order to overcome the shortcomings of existing technologies such as excessive manual intervention, low efficiency, and difficulty in parsing multimodal inputs, and to achieve intelligent identification, automatic classification, and accurate allocation of citizen issues, thereby improving the level of intelligence and convenience of government services.

[0009] The technical objective of this invention is achieved as follows: a method for intelligent allocation and processing of government affairs events based on a multimodal large model, the specific method of which is as follows:

[0010] Multimodal data acquisition and preprocessing: Receive and parse modal information of text, voice and images submitted from government affairs clients, construct an initial event corpus and complete data cleaning and structuring to obtain preprocessed multimodal data;

[0011] Multimodal semantic fusion and intent recognition: The DeepSeek-R1 model is used to perform multimodal semantic fusion on the preprocessed multimodal data to identify the user's core intent and event content;

[0012] Event classification and responsible department matching: The structured event semantics are matched with the knowledge graph of government agencies to identify the corresponding processing unit or responsible department and assign tasks.

[0013] GAT Path Reasoning: Obtain the relevance score between each department and the event through graph convolutional layer (GCN), graph attention layer (GAT) and attention mechanism, and designate the department with the highest score as the responsible department;

[0014] Intelligent work order generation and cross-departmental collaboration: Automatically generate processing work orders based on the identification results, and complete responsibility assignment, collaborative tracking and lifecycle management;

[0015] User feedback and closed-loop optimization: Feedback of processing results to users and collection of user feedback information for system self-optimization and iteration.

[0016] As a preferred method, the multimodal data acquisition and preprocessing are as follows:

[0017] Image enhancement: The uploaded image is enhanced in two domains (spatial domain + frequency domain) to improve image sharpness, remove noise and segment the region; specifically: first, five layers of convolution + ReLUctant are used to improve sharpness, then wavelet swap is used for x3 noise suppression, and finally, a fine segmentation map is generated by U-Net to extract the scene ROI;

[0018] OCR Recognition: The Tesseract+ self-trained Chinese OCR model is used to recognize text information in images and key content fields are extracted by combining location features;

[0019] Key object extraction: Based on the Swin-Transformer-Tiny pre-trained + Fine-Tune model, identify entity elements of traffic signs, government documents, and posters in the scene;

[0020] Speech transcription and voice emotion feature extraction: The DeepSeek-Voice ASR system (speech recognition system) is called to transcribe speech information into text in real time. At the same time, 16-dimensional MFCC, pitch, and energy features are extracted, and a 2-layer LSTM model is used to output a voice emotion distribution probability vector. The voice emotion distribution probability vector includes speaker features (age, gender) and tone emotion, which are used for subsequent emotion judgment.

[0021] Metadata binding: Each input record is tagged with a timestamp (millisecond level), user ID, latitude and longitude (GPS), and device ID, and uniformly transferred to the event corpus database to construct a structured initial corpus; the schema fields in the structured initial corpus include event_id, timestamp, user_id, modality, raw_data, geo, and preprocessed_data;

[0022] Preprocessing: After the initial data collection is completed, the collected data will be cleaned and fused to ensure the consistency and accuracy of the input data; and the collection results will be standardized, specifically: text input will be uniformly encoded as UTF-8; images and audio will be deduplicated using MD5 signatures; and field integrity check rules will be set.

[0023] As a preferred approach, multimodal semantic fusion and intent recognition are specifically as follows:

[0024] Unified Semantic Encoding: Inputs from different modalities are projected onto a unified 512-dimensional semantic space, concatenated, and with positional embeddings added before being fed into a cross-modal Transformer layer (8 layers, 12 attention heads, dropout = 0.1) to ensure comprehensive analysis of data from different modalities at the same semantic level. Each modal input is processed by a corresponding Encoder: Text SeqEncoder: A Bi-LSTM model is used to process the text, combined with attention layers to capture long-range semantic information; Image Encoder: Cross-modal fusion technology using ResNet50 and SwinTransformer is used to further process image data, ensuring that image information can be effectively combined with other modal information; Speech Encoder: A 3-layer CNN + 2-layer LSTM encoding of vocal features is used to extract vocal features for subsequent sentiment analysis.

[0025] Multimodal alignment training: A multimodal alignment loss function is introduced to better align modalities such as images, text, and speech, strengthening the semantic consistency of visual-text-speech representation and improving robustness and accuracy under semantic ambiguity conditions. The multimodal alignment loss function takes the following form:

[0026]

[0027] Among them, v i , t i a i These are the image, text, and speech encoding vectors for the same event i, respectively.

[0028] Weighted Consolidation of Main Task Losses (e.g., cross-entropy for intentional classification) and alignment loss, the formulas are as follows:

[0029]

[0030] λ is initially set to 0.5 and can be dynamically adjusted; Modality Dropout (random masking ratio of 0.2 for each modality) is introduced during the training phase to enhance robustness;

[0031] Intent Extraction: Combining historical corpora and multi-turn dialogue content, the core user intent is identified through a long-range attention mechanism. Key elements of the event are extracted through a classifier and an information extractor, including event type, target audience, time of occurrence, location, and service category. Entity extraction is then used to accurately identify the core requirements of the event, enabling subsequent event classification and responsibility matching. Specifically, the vector output from the Transformer layer is further fed into the classifier and information extractor: entity extraction is achieved using CRF+Bi-LSTM; the intent classifier is a 3-layer DNN (hidden: 512→256→numClasses), with softmax outputting the event type.

[0032] Structured semantic output: Finally, a structured event object is generated, along with structured semantic tags.

[0033] As a preferred approach, the specific details of event classification and responsibility matching are as follows:

[0034] Constructing a government knowledge graph: A government knowledge graph containing entities and edges of "item name - department function - administrative division - hierarchical relationship" is constructed, and a high-dimensional representation of the nodes in the government knowledge graph is built using a graph neural network (GNN). The features of the government knowledge graph nodes include department functions, processing timeliness, and historical event data. A hierarchical construction strategy is adopted to ensure the dynamic expansion and updating of the government knowledge graph. The government knowledge graph undergoes incremental learning monthly, maintaining its accuracy by adding data, modifying functional descriptions, or adjusting departmental relationships.

[0035] Node Encoding: Graph Neural Networks (GNNs) are used to encode each node in the government knowledge graph. Specifically: Node Feature Processing: Each node represents a government department or matter. The features of each node are initialized through vectorized text descriptions and historical data (such as processing time); GNN Encoding: Graph Attention Networks (GAT) models are used to represent nodes in high-dimensional vector form. The vector representation of a node can reflect the function, geographical location, and historical processing capabilities of the corresponding node.

[0036] Path matching and reasoning: The GAT (Graph Attention Network) model is used for path reasoning to identify the responsible department for handling events. The GAT model introduces a graph attention mechanism to calculate weighted relationships between nodes. The path reasoning algorithm finds the optimal responsible unit through multi-hop paths. Specifically: Path reasoning: In the government knowledge graph, from the "geographical jurisdiction" of the event to the "item keywords" and then to the "responsible department," the GAT model performs multi-hop reasoning to discover the optimal handling department. Reasoning process: The GAT model infers the confidence of the path by focusing on the relationships between nodes in the government knowledge graph (such as historical processing timeliness, departmental functions, etc.). Finally, based on the path weights, the most suitable responsible department is selected.

[0037] Ranking of Responsible Entities: Multiple matching candidate units are scored and ranked, taking into account factors such as edge weights between nodes, historical processing timeliness, and user evaluations. The department with the highest overall score will become the responsible entity for the incident. Among them, edge weights (weights in the graph) include the department's historical processing timeliness and service quality; historical timeliness means that departments with fast processing speeds will be selected first; user evaluations refer to the assessment of the corresponding department's processing capabilities based on user feedback on the corresponding department's historical processing results.

[0038] Fuzzy classification processing: In cases of fuzzy expressions or polysemous words, deep semantic matching and prior event statistical models are combined to improve the accuracy of responsibility division, and ambiguity is automatically identified and eliminated by comparing historical event data.

[0039] As a preferred approach, GAT path reasoning is as follows:

[0040] Graph Convolutional Layer (GCN): In a graph convolutional layer, node features are updated based on information from their neighboring nodes; the node representation of each layer is calculated by weighted averaging of information from its neighboring nodes, and the updated node representation contains richer contextual information.

[0041] Graph Attention Layer (GAT): Calculates the attention weights between nodes. These attention weights are used to focus on important nodes and edges in the graph, thereby improving the accuracy of inference. Each GAT layer calculates a weighted average of the nodes and performs multi-hop inference based on the structure of the graph.

[0042] Attention mechanism: By learning the relationships and semantic distances between nodes, the attention between nodes is automatically adjusted, thereby improving the accuracy of event responsibility matching.

[0043] As a preferred approach, intelligent work order generation and cross-departmental collaboration are specifically implemented as follows:

[0044] Create work order templates: Generate standardized work order templates based on the structured semantics of events; each work order includes fields such as problem description, responsible department, event type, response level, time limit requirement, and warning status;

[0045] Cross-departmental work order splitting: This process handles collaborative events involving multiple responsible parties, generates sub-task assignment orders, and dynamically assigns sub-work orders using a responsibility decomposition tree (RDB). Specifically, when an event involves multiple departments, the work order is split to ensure each responsible department receives its relevant portion of the work order. The core of cross-departmental work order splitting lies in the reasonable decomposition of tasks based on the RDB algorithm, ensuring collaboration between departments. The RDB algorithm is used to split work orders across multiple departments and assign specific tasks, specifically: Task decomposition: Based on event type, relevant departmental functions, and processing time, the work order is split into multiple sub-tasks; Decomposition rules: Each sub-task is handled by one or more responsible departments, and each department is only responsible for a portion of the work within its corresponding functional scope; Upstream and downstream relationships: Sub-tasks may depend on the completion of other sub-tasks, such as a repair task depending on a preceding review task.

[0046] Work order status management: Track the status transition of work orders. Work order status includes: "Unprocessed → Accepted → Processing Completed → User Feedback → Archived";

[0047] API Integration and Dispatch: Supports interaction with mainstream government platforms (such as the 12345 hotline platform and government cloud business systems) via RESTful API to accurately dispatch work orders to business systems;

[0048] Blockchain auditing records every step of the work order's dispatch, processing, and feedback on the blockchain, enabling traceability, tamper-proofing, and automatic evidence storage.

[0049] More specifically, user feedback and closed-loop optimization are as follows:

[0050] Results push: Push the processing completion result to the user's client (App, SMS, etc.) and display the processor, processing progress and processing basis;

[0051] Satisfaction evaluation interface: guides users to evaluate the processing effect and collects subjective ratings and text opinions;

[0052] User behavior analysis: Construct feedback tags based on user clicks, viewing paths, and frequency of follow-up questions;

[0053] RLHF Training Mechanism: This mechanism introduces Human Feedback Reinforcement Learning (RLHF) to fine-tune the DeepSeek model using satisfaction ratings as the reward function, optimizing its intent understanding, responsibility matching, and question-answer generation capabilities. Specifically: The RLHF fine-tuning mechanism uses user satisfaction ratings as reward signals to optimize model performance. By introducing the RLHF strategy, it dynamically adjusts model parameters to improve the accuracy of event intent, department matching, and question-answer generation. Reward Function Design: The RLHF fine-tuning mechanism is trained using the following reward function: R = α·Satisfaction Rating + β·Processing Time + γ·Event Matching Degree ; where α, β, and γ are weighting coefficients, which are adjusted experimentally to ensure a balance between user satisfaction, timeliness, and event matching; dynamic strategy optimization: based on feedback data, the intent classifier parameters, multimodal fusion weights, and responsibility allocation path selection rules are automatically updated to ensure the system can self-optimize and continuously improve service quality; dynamic model and strategy updates: based on user feedback and real-time event data, the model and strategy will be automatically updated through incremental learning, online update mechanisms, adaptive adjustments, and seamless deployment of model updates to ensure the system can continuously improve and adapt to new challenges without restarting.

[0054] A government affairs event intelligent allocation and processing system based on a multimodal large model, the system comprising:

[0055] The multimodal data acquisition and preprocessing module is used to receive and parse modal information of text, voice and images submitted from government clients (APP, web page, hotline, etc.), construct an initial event corpus, and complete data cleaning and structuring processing.

[0056] The multimodal semantic fusion and intent recognition module is used to map multimodal inputs of text, images and voice to a unified semantic representation space, and to identify the user's true needs, core intents and event context elements;

[0057] The event classification and responsible department matching module is used to match structured event semantics with the knowledge graph of government agencies to identify the corresponding processing unit or responsible department;

[0058] The intelligent work order generation and cross-departmental collaboration module is used to automatically generate processing work orders based on the recognition results, and complete responsibility assignment, collaborative tracking and lifecycle management;

[0059] The user feedback and closed-loop optimization module is used to feed back the processing results to the user and collect their feedback information, which is then used for the system's self-optimization and iteration.

[0060] Preferably, the multimodal data acquisition and preprocessing module includes:

[0061] The image enhancement submodule is used to improve image sharpness, remove noise, and segment regions of uploaded images.

[0062] The OCR recognition submodule is used to recognize text information in images and extract key content fields by combining location features;

[0063] The key object extraction submodule is used to identify entity elements in a scene based on the image recognition model (Swin Transformer), such as traffic signs, identity information, and tables.

[0064] The speech transcription and analysis submodule is used to call the speech recognition engine (DeepSeek-Voice) to transcribe speech into text in real time, and extract speaker features (such as age and gender) and tone and emotion features for subsequent emotion judgment.

[0065] The timestamp annotation and user identifier association sub-unit is used to bind metadata such as event submission time, user ID and location coordinates to each input modality information to build a preliminary event context;

[0066] The multimodal semantic fusion and intent recognition module includes:

[0067] The unified semantic encoding submodule is used to use DeepSeek-R1 to fuse Transformer architecture models and project different modal inputs into a shared semantic vector space.

[0068] The context-aware parsing submodule is used to combine historical corpora and multi-turn dialogue content to identify the user's core intent through a long-distance attention mechanism.

[0069] The semantic fusion submodule is used to introduce a multimodal alignment loss function to enhance the consistent expression of visual-text-speech semantics and improve robustness and accuracy under semantic ambiguity conditions.

[0070] The event element extraction submodule is used to extract key information fields, including event type, target audience, time of occurrence, location, and service category.

[0071] The structured semantic output submodule is used to build structured semantic tags.

[0072] Even better, the event classification and responsible department matching module includes:

[0073] The government knowledge graph construction submodule is used to build an information graph containing entities and edges such as "item name - department function - administrative division - hierarchical relationship", which supports dynamic updates and incremental learning;

[0074] The node encoding submodule is used to perform high-dimensional vectorized representation of graph nodes using a graph neural network (GNN);

[0075] The path matching submodule is used to introduce multi-hop semantic path reasoning based on graph attention network (GAT) to find the most confident responsible entity node in the path of "geographic jurisdiction → subject keyword → responsible department".

[0076] The Responsibility Entity Ranking Submodule is used to score and rank multiple matching candidate units, taking into account factors such as edge weights between nodes, historical processing timeliness, and user evaluation.

[0077] The fuzzy classification processing submodule is used to improve the accuracy of responsibility division by combining deep semantic matching and prior event statistical models in cases of fuzzy expressions or polysemous words.

[0078] The intelligent work order generation and cross-departmental collaboration module includes:

[0079] The work order template construction submodule is used to map structured semantics to a standardized work order structure. The fields of the standardized work order structure include problem description, responsible department, event type, response level, time limit requirement, and warning status.

[0080] The work order status management submodule is used to track the status transition of work orders, including: "Unprocessed → Accepted → Processing Completed → User Feedback → Archived";

[0081] The API integration and dispatch submodule is used to support interaction with mainstream government platforms (such as the 12345 hotline platform and government cloud business system) through RESTful API, and to accurately dispatch work orders to the business system;

[0082] The blockchain auditing sub-module is used to record each step of the work order operation (such as transfer, processing, and feedback) on the blockchain to achieve traceability, tamper-proof, and automatic evidence storage.

[0083] A cross-departmental collaboration engine is used to handle collaborative events involving multiple responsible parties, generate sub-task assignment orders, and dynamically assign sub-work orders through a responsibility decomposition tree.

[0084] The user feedback and closed-loop optimization module includes:

[0085] The results push submodule is used to push the processing completion results to the user's client (App, SMS, etc.) and display the processor, processing progress and processing basis;

[0086] The satisfaction evaluation interface submodule is used to guide users to evaluate the processing effect and collect subjective ratings and text opinions;

[0087] The user behavior analysis submodule is used to build feedback tags based on user clicks, viewing paths, and frequency of follow-up questions.

[0088] The RLHF training mechanism submodule is used to introduce the "Human Feedback Reinforcement Learning (RLHF)" strategy, which uses satisfaction as a reward function to fine-tune the DeepSeek model and optimize its intent understanding, responsibility matching, and question answer generation capabilities.

[0089] The dynamic strategy optimization submodule is used to automatically update the intent classifier parameters, multimodal fusion weights, and responsibility allocation path selection rules based on feedback data.

[0090] The intelligent allocation and processing method and system for government affairs based on a multimodal large model of the present invention has the following advantages:

[0091] (i) Compared with existing technologies, this invention improves event processing efficiency, realizing a closed-loop process of automatic reception, parsing, classification, transfer, and feedback; improves recognition accuracy by adapting to complex content expression forms through the multimodal understanding capabilities of large models; enhances allocation accuracy by using semantic alignment and knowledge graphs to locate responsible departments and reduce misassignment and omission; strengthens cross-departmental collaborative processing mechanisms to promote the automation and networking of government service processes; improves citizen satisfaction, provides a good user experience, and has scalability and large-scale deployment value.

[0092] (II) This invention utilizes the semantic analysis, image recognition, and speech processing capabilities of a multimodal large model, combined with knowledge graph technology, to achieve intelligent classification and allocation of government affairs events, thereby improving processing efficiency, optimizing the citizen service experience, and solving the problems of excessive manual intervention, low efficiency, and difficulty in parsing multimodal inputs in existing technologies.

[0093] (III) This invention achieves event classification and matching of responsible departments by connecting with government knowledge graphs. By matching key event information with elements such as departmental functions and geographical information in the knowledge graph, the responsible department for the event can be automatically identified and assigned. Attached Figure Description

[0094] The invention will be further described below with reference to the accompanying drawings.

[0095] Appendix Figure 1 The flowchart is for a method of intelligent allocation and processing of government affairs events based on a multimodal large model.

[0096] Appendix Figure 2 This is a flowchart of the model fine-tuning process using human feedback reinforcement learning (RLHF). Detailed Implementation

[0097] The following detailed description of the intelligent allocation and processing method and system for government affairs based on a multimodal large model, with reference to the accompanying drawings and specific embodiments, is provided in the specification.

[0098] Example 1:

[0099] As attached Figure 1 As shown in the figure, this embodiment provides a method for intelligent allocation and processing of government affairs events based on a multimodal large model. The method is as follows:

[0100] S1. Multimodal data acquisition and preprocessing: Receive and parse modal information of text, voice and images submitted from government affairs clients, construct an initial event corpus and complete data cleaning and structuring to obtain preprocessed multimodal data;

[0101] S2. Multimodal Semantic Fusion and Intent Recognition: The DeepSeek-R1 model is called to perform multimodal semantic fusion on the preprocessed multimodal data to identify the user's core intent and event content;

[0102] S3. Event Classification and Responsible Department Matching: Match structured event semantics with government agency knowledge graphs to identify the corresponding processing units or responsible departments and assign tasks.

[0103] S4, GAT Path Reasoning: Obtain the relevance score between each department and the event through graph convolutional layer (GCN), graph attention layer (GAT) and attention mechanism, and take the department with the highest score as the responsible department;

[0104] S5. Intelligent work order generation and cross-departmental collaboration: Automatically generate processing work orders based on the identification results, and complete responsibility assignment, collaborative tracking and lifecycle management;

[0105] S6. User Feedback and Closed-Loop Optimization: Feedback the processing results to the user and collect user feedback information for system self-optimization and iteration.

[0106] The multimodal data acquisition and preprocessing in step S1 of this embodiment are as follows:

[0107] S101. Image Enhancement: The uploaded image is enhanced in terms of image sharpness, noise removal and region segmentation using a dual-domain (spatial domain + frequency domain) approach. Specifically, five layers of convolution + ReLUctant are used to improve sharpness, then wavelet swap is used for x3 noise suppression, and finally U-Net is used to generate a fine segmentation map to extract the scene ROI and improve image quality. Especially in cases where the image is blurry or contains a complex background, key information in the image can be clearly extracted.

[0108] S102, OCR Recognition: The Tesseract+ self-trained Chinese OCR model is used to recognize text information in images. The text recognition rate of the enhanced image reaches 98%, and key content fields such as "address" and "description" are extracted by combining location features.

[0109] S103, Key Object Extraction: Based on the Swin-Transformer-Tiny pre-trained + Fine-Tune model, entity elements of traffic signs, government documents and posters in the scene are identified, with an average mAP of 0.87 and a model input resolution of 640×640.

[0110] S104. Speech Transcription and Voice Feature Extraction: The DeepSeek-Voice ASR system (speech recognition system) is called to transcribe speech information into text in real time, with a WER (word error rate) of ≤6%. At the same time, 16-dimensional MFCC, pitch, and energy features are extracted, and a 2-layer LSTM model is used to output a voice emotion distribution probability vector. The voice emotion distribution probability vector includes speaker features (age, gender) and tone emotion, which are used for subsequent emotion judgment.

[0111] S105. Metadata Binding: Add a timestamp (millisecond level), user ID, latitude and longitude (GPS), and device ID to each input record, and uniformly transfer them to the event corpus database to build a structured initial corpus. This step takes approximately <500ms. The schema fields in the structured initial corpus include event_id, timestamp, user_id, modality, raw_data, geo, and preprocessed_data.

[0112] S106. Preprocessing: After the initial data collection is completed, the collected data will be cleaned and fused to ensure the consistency and accuracy of the input data; and the collection results will be standardized, specifically: text input will be uniformly encoded as UTF-8; images and audio will be deduplicated using MD5 signatures; and field integrity check rules will be set, such as ensuring that the difference in location coordinates is less than 100m when images and audio are submitted at the same time, otherwise a prompt for user confirmation will be triggered.

[0113] The multimodal semantic fusion and intent recognition in step S2 of this embodiment are as follows:

[0114] S201, Unified Semantic Coding: Inputs from different modalities are projected onto a unified 512-dimensional semantic space, concatenated, and with positional embeddings added before being fed into a cross-modal Transformer layer (8 layers, 12 attention heads, dropout = 0.1) to ensure comprehensive analysis of data from different modalities at the same semantic level. Each modal input is processed by a corresponding Encoder: Text SeqEncoder: A Bi-LSTM model is used to process the text, combined with attention layers to capture long-range semantic information; Image Encoder: Cross-modal fusion technology using ResNet50 and SwinTransformer is used to further process image data, ensuring that image information can be effectively combined with other modal information; Speech Encoder: A 3-layer CNN + 2-layer LSTM encoding of vocal features is used to extract vocal features for subsequent sentiment analysis.

[0115] S202, Multimodal Alignment Training: A multimodal alignment loss function is introduced to better align modalities such as image, text, and speech, strengthening the semantic consistency of visual-text-speech representation and improving robustness and accuracy under semantic ambiguity conditions. The multimodal alignment loss function takes the following form:

[0116]

[0117] Among them, v i , t i a i These are the image, text, and speech encoding vectors for the same event i, respectively.

[0118] Weighted Consolidation of Main Task Losses (e.g., cross-entropy for intentional classification) and alignment loss, the formulas are as follows:

[0119]

[0120] The initial value of λ is set to 0.5 and can be dynamically adjusted. Modality Dropout (with a random masking ratio of 0.2 for each modality) is introduced during the training phase to enhance robustness. The training dataset consists of 100,000 labeled event data.

[0121] S203. Intent Extraction: Combining historical corpora and multi-turn dialogue content, the core intent of the user is identified through a long-distance attention mechanism. Key elements of the event are extracted using a classifier and information extractor, including event type, target audience, time of occurrence, location, and service category. Entity extraction accurately identifies the core requirements of the event, leading to subsequent event classification and responsibility matching. Specifically, the vector output from the Transformer layer is further fed into the classifier and information extractor. CRF+Bi-LSTM is used for entity extraction, achieving an F1 score of over 0.92 for named entity recognition. The intent classifier is a 3-layer DNN (hidden: 512→256→numClasses), with softmax outputting the event type (e.g., "business license change"), achieving an accuracy of ≥93%.

[0122] S204. Structured Semantic Output: Finally, a structured event object is generated, along with structured semantic tags, in the following form:

[0123] {

[0124] "type":"One thing for businesses",

[0125] "intent":"Issuance of business license",

[0126] "location":"someone",

[0127] "time":"2025-06-29T09:12:34",

[0128] "entity_list":[..."Newly Established Companies","Business Licenses"...],

[0129] "description": "User recorded voice message: 'I want to apply to open a company. How do I apply for a business license? I hope this can be processed promptly.'"

[0130] }

[0131] The event classification and responsibility matching in step S3 of this embodiment are as follows:

[0132] S301. Constructing a Government Knowledge Graph: Construct a government knowledge graph containing entities and edges of "item name - department function - administrative division - hierarchical relationship", and construct a high-dimensional representation of the government knowledge graph nodes through a graph neural network (GNN); the features of the government knowledge graph nodes include department functions, processing timeliness, and historical event data; at the same time, a hierarchical construction strategy is adopted to ensure the dynamic expansion and updating of the government knowledge graph. The government knowledge graph undergoes incremental learning every month, and the accuracy of the government knowledge graph is maintained by adding data, modifying functional descriptions, or adjusting departmental relationships;

[0133] S302, Node Encoding: Graph Neural Networks (GNNs) are used to encode each node in the government knowledge graph. Specifically: Node Feature Processing: Each node represents a government department or matter. The features of each node are initialized through vectorized text descriptions and historical data (such as processing time); GNN Encoding: Graph Attention Networks (GAT) models are used to represent nodes in high-dimensional vector form. The vector representation of a node can reflect the function, geographical location, and historical processing capabilities of the corresponding node.

[0134] S303, Path Matching and Reasoning: The GAT (Graph Attention Network) model is used for path reasoning to identify the responsible department for handling events. The GAT model introduces a graph attention mechanism to calculate weighted relationships between nodes. The path reasoning algorithm finds the optimal responsible unit through multi-hop paths. Specifically: Path Reasoning: In the government knowledge graph, the GAT model performs multi-hop reasoning from the event's "geographical jurisdiction" to "event keywords" and then to the "responsible department," thereby identifying the optimal handling department. Reasoning Process: The GAT model infers the confidence of the path by focusing on the relationships between nodes in the government knowledge graph (such as historical processing timeliness, departmental functions, etc.). Finally, based on the path weights, the most suitable responsible department is selected.

[0135] S304. Ranking of Responsible Entities: Multiple matching candidate units are scored and ranked, taking into account factors such as edge weights between nodes, historical processing timeliness, and user evaluations. The department with the highest overall score will become the responsible entity for the event. Among them, edge weights (weights in the graph) include the department's historical processing timeliness and service quality; historical timeliness means that departments with fast processing speeds will be selected first; user evaluations refer to the assessment of the corresponding department's processing capabilities based on user feedback on the corresponding department's historical processing results.

[0136] S305. Fuzzy Classification Processing: In cases of fuzzy expressions or polysemous words, combine deep semantic matching with prior event statistical models to improve the accuracy of responsibility division, and automatically identify and eliminate ambiguity by comparing historical event data.

[0137] The GAT path reasoning in step S4 of this embodiment is as follows:

[0138] S401, Graph Convolutional Layer (GCN): In a graph convolutional layer, node features are updated based on the information of their neighboring nodes; the node representation of each layer is calculated by weighted averaging of the information of its neighboring nodes, and the updated node representation contains richer contextual information.

[0139] S402, Graph Attention Layer (GAT): Calculates the attention weights between nodes. These attention weights are used to focus on important nodes and edges in the graph, thereby improving the accuracy of inference. Each GAT layer calculates the weighted average of the nodes and performs multi-hop inference based on the structure of the graph.

[0140] S403 Attention Mechanism: By learning the relationships and semantic distances between nodes, the attention between nodes is automatically adjusted, thereby improving the accuracy of event responsibility matching; during the reasoning process, the parameters of path reasoning are continuously optimized to ensure the efficiency and accuracy of the path reasoning model.

[0141] The intelligent work order generation and cross-departmental collaboration in step S5 of this embodiment are as follows:

[0142] S501. Constructing Work Order Templates: Generate standardized work order templates based on the structured semantics of events; each work order includes fields such as problem description, responsible department, event type, response level, time limit, and warning status; among which, problem description: generate a concise and clear problem description based on the event description; responsible department: fill in according to the results of the responsible department matching module; event type: for example, "municipal road damage repair"; response level and time limit: set according to the urgency of the event and the time limit requirements;

[0143] S502, Cross-departmental Work Order Splitting: This process handles collaborative events involving multiple responsible parties, generates sub-task assignment orders, and dynamically assigns sub-work orders using a responsibility decomposition tree (RDB). Specifically, when an event involves multiple departments, the work order is split to ensure each responsible department receives its relevant portion of the work order. The core of cross-departmental work order splitting lies in the reasonable decomposition of tasks based on the RDB algorithm, ensuring collaboration between departments. The RDB algorithm is used to split work orders across multiple departments and assign specific tasks, specifically: Task Decomposition: Based on event type, relevant departmental functions, and processing time, the work order is split into multiple sub-tasks; Decomposition Rules: Each sub-task is handled by one or more responsible departments, and each department is only responsible for a portion of the work within its corresponding functional scope; Upstream and Downstream Relationships: Sub-tasks may depend on the completion of other sub-tasks, such as a repair task depending on a preceding review task.

[0144] Among them, the pseudocode for Responsibility Decomposition Tree (RBT) is as follows:

[0145] Example of a splitting strategy: Suppose there is a cross-departmental event: "Municipal Road Damage Repair", involving departments including "Municipal Engineering Department", "Traffic Management Department" and "Public Safety Department"; Municipal Engineering Department is responsible for road repair; Traffic Management Department is responsible for traffic control and traffic signal adjustment; Public Safety Department is responsible for site safety; after the work order is split, a responsibility decomposition tree will be generated, with each department responsible for different sub-tasks;

[0146] S503, Work Order Status Management: Tracks the transition of work order status. Work order status includes: "Unprocessed → Accepted → Processed → User Feedback → Archived"; where Unprocessed: The work order has been generated and is waiting for assignment; Accepted: The responsible department has received and started processing; Processed: The problem has been resolved and is awaiting user confirmation; User Feedback: Feedback is recorded after user confirmation; Archived: The work order is closed and archived.

[0147] S504, API Integration and Dispatch: Supports interaction with mainstream government platforms (such as the 12345 hotline platform and government cloud business systems) via RESTful API, accurately distributing work orders to business systems;

[0148] S505 Blockchain Audit: Record every step of the work order's dispatch, processing, and feedback on the blockchain to achieve traceability, tamper-proofing, and automatic evidence storage.

[0149] The user feedback and closed-loop optimization in step S6 of this embodiment are as follows:

[0150] S601. Result Push: Push the processing completion result to the user's client (App, SMS, etc.) and display the processor, processing progress and processing basis;

[0151] S602, Satisfaction Evaluation Interface: Guides users to evaluate the processing effect and collects subjective ratings and textual opinions;

[0152] S603, User Behavior Analysis: Construct feedback tags based on user clicks, viewing paths, and frequency of follow-up questions;

[0153] S604 and RLHF training mechanisms: as shown in the appendix. Figure 2As shown, a Human Feedback Reinforcement Learning (RLHF) strategy is introduced to fine-tune the DeepSeek model using satisfaction as the reward function, optimizing its intent understanding, responsibility matching, and question-answer generation capabilities. Specifically: The RLHF fine-tuning mechanism uses user satisfaction ratings as reward signals to optimize model performance. By introducing the RLHF strategy, the model parameters are dynamically adjusted to improve the accuracy of event intent, department matching, and question-answer generation. The reward function design uses the following reward function for training: R = α·satisfaction rating + β·processing timeliness + γ·event matching degree; where... α, β, and γ are weighting coefficients, which are adjusted experimentally to ensure a balance between user satisfaction, timeliness, and event matching. Dynamic strategy optimization: Based on feedback data, the intent classifier parameters, multimodal fusion weights, and responsibility allocation path selection rules are automatically updated to ensure the system can self-optimize and continuously improve service quality. Dynamic model and strategy updates: Based on user feedback and real-time event data, the model and strategy are automatically updated through incremental learning, online update mechanisms, adaptive adjustments, and seamless deployment of model updates, ensuring the system can continuously improve and adapt to new challenges without restarting.

[0154] Based on user feedback and real-time event data, the system will automatically update its models and strategies. This process is achieved through incremental learning and online update mechanisms, ensuring that the system can continuously improve and adapt to new challenges without restarting; specifically as follows:

[0155] (1) Incremental learning strategy: Incremental learning is a method that allows the model to be updated when new data arrives without having to start training from scratch; the core of this strategy is to save the existing state of the model and fine-tune it using new data, ensuring that the model continuously absorbs new knowledge while maintaining historical knowledge; as follows:

[0156] ① Incremental Data Acquisition: The system receives new event data in real time and generates new training samples based on user feedback and event processing results. This data will be added to the training set periodically as incremental data.

[0157] ② Update frequency: The frequency of incremental learning is dynamic and is adjusted based on factors such as system load and data changes. For example, the system may perform incremental learning once every hour or after processing a certain amount of data.

[0158] ③ Parameter fine-tuning: Model parameters are fine-tuned through small adjustments. The specific method is to train the model using newly collected data and update its weights, usually using a small learning rate (such as 1e-5 or 1e-6) to avoid losing previously learned knowledge.

[0159] (2) Online update mechanism: Ensures real-time optimization of the system, enabling updates to models and strategies based on real-time data during runtime; the advantage of online updates lies in their ability to flexibly handle changing data and task environments without interrupting system services; details are as follows:

[0160] ① Data Inflow: Whenever new event data or user feedback enters the system, it is first cached and preprocessed in real time. The processed data is then passed to the model as an "increment".

[0161] ② Model Update: The model is quickly adjusted using online learning algorithms (online gradient descent, incremental learning of neural networks). Typically, updates do not immediately affect online services; the update process is completed in the background, ensuring that the updated model seamlessly replaces the old model.

[0162] ③ Feedback Mechanism: After collecting user feedback in real time, the system can dynamically adjust the model's strategy based on the feedback. For example, if the processing results for certain event categories are poor, the model will automatically adjust to improve the processing capability for those categories. User satisfaction ratings and processing timeliness will be key factors in the dynamic update process.

[0163] (3) Adaptive strategy adjustment: This aims to automatically adjust the system's working strategy based on the processing effect of real-time events and user feedback. This refers not only to adjustments at the algorithm level, but also to optimizations in business processes and task scheduling, as detailed below:

[0164] ① Strategy Change Detection: The system periodically monitors key performance indicators (KPIs) during the processing, such as task completion time and user satisfaction. When deviations in certain indicators are detected, the system will attempt to automatically adjust the strategy and optimize the processing flow.

[0165] ② Strategy optimization and adjustment: For example, when the processing efficiency of some departments declines, the system will automatically modify the workflow, optimize the work order assignment strategy, and assign tasks to departments with stronger processing capabilities; or adjust task priority and timeliness based on historical processing data.

[0166] ③ Reinforcement Learning Mechanism: Through Deep Q-Learning, the system continuously updates and optimizes its strategy based on long-term user feedback. The model evaluates the effectiveness of the current strategy by constantly receiving new environmental feedback and adjusts the strategy accordingly.

[0167] (4) Seamless deployment of model updates: To ensure that model updates do not affect the system's operating efficiency, a seamless deployment scheme is adopted, as follows:

[0168] ① Hot update: Through the hot update mechanism, the new model version is loaded directly without restarting the service, and the microservice architecture is used to ensure the transparency and non-interference of the update process.

[0169] ② Version Control: A version control mechanism is used to ensure that each update is meticulously recorded and can be rolled back to previous versions in case of problems. Each new model after incremental learning is marked as a new version, and historical versions are archived.

[0170] Example 2:

[0171] This embodiment provides an intelligent allocation and processing system for government affairs events based on a multimodal large model. The system includes:

[0172] The multimodal data acquisition and preprocessing module is used to receive and parse modal information of text, voice and images submitted from government clients (APP, web page, hotline, etc.), construct an initial event corpus, and complete data cleaning and structuring processing.

[0173] The multimodal semantic fusion and intent recognition module is used to map multimodal inputs of text, images and voice to a unified semantic representation space, and to identify the user's true needs, core intents and event context elements;

[0174] The event classification and responsible department matching module is used to match structured event semantics with the knowledge graph of government agencies to identify the corresponding processing unit or responsible department;

[0175] The intelligent work order generation and cross-departmental collaboration module is used to automatically generate processing work orders based on the recognition results, and complete responsibility assignment, collaborative tracking and lifecycle management;

[0176] The user feedback and closed-loop optimization module is used to feed back the processing results to the user and collect their feedback information, which is then used for the system's self-optimization and iteration.

[0177] The multimodal data acquisition and preprocessing module in this embodiment includes:

[0178] The image enhancement submodule is used to improve image sharpness, remove noise, and segment regions of uploaded images.

[0179] The OCR recognition submodule is used to recognize text information in images and extract key content fields by combining location features;

[0180] The key object extraction submodule is used to identify entity elements in a scene based on the image recognition model (Swin Transformer), such as traffic signs, identity information, and tables.

[0181] The speech transcription and analysis submodule is used to call the speech recognition engine (DeepSeek-Voice) to transcribe speech into text in real time, and extract speaker features (such as age and gender) and tone and emotion features for subsequent emotion judgment.

[0182] The timestamp annotation and user identifier association sub-unit is used to bind metadata such as event submission time, user ID, and location coordinates to each input modality information to construct a preliminary event context.

[0183] The multimodal semantic fusion and intent recognition module in this embodiment includes:

[0184] The unified semantic encoding submodule is used to use DeepSeek-R1 to fuse Transformer architecture models and project different modal inputs into a shared semantic vector space.

[0185] The context-aware parsing submodule is used to combine historical corpora and multi-turn dialogue content to identify the user's core intent through a long-distance attention mechanism.

[0186] The semantic fusion submodule is used to introduce a multimodal alignment loss function to enhance the consistent expression of visual-text-speech semantics and improve robustness and accuracy under semantic ambiguity conditions.

[0187] The event element extraction submodule is used to extract key information fields, including event type, target audience, time of occurrence, location, and service category.

[0188] The structured semantic output submodule is used to build structured semantic tags.

[0189] The event classification and responsible department matching module in this embodiment includes:

[0190] The government knowledge graph construction submodule is used to build an information graph containing entities and edges such as "item name - department function - administrative division - hierarchical relationship", which supports dynamic updates and incremental learning;

[0191] The node encoding submodule is used to perform high-dimensional vectorized representation of graph nodes using a graph neural network (GNN);

[0192] The path matching submodule is used to introduce multi-hop semantic path reasoning based on graph attention network (GAT) to find the most confident responsible entity node in the path of "geographic jurisdiction → subject keyword → responsible department".

[0193] The Responsibility Entity Ranking Submodule is used to score and rank multiple matching candidate units, taking into account factors such as edge weights between nodes, historical processing timeliness, and user evaluation.

[0194] The fuzzy classification processing submodule is used to improve the accuracy of responsibility division by combining deep semantic matching and prior event statistical models in cases of fuzzy expressions or polysemous words.

[0195] The intelligent work order generation and cross-departmental collaboration module in this embodiment includes:

[0196] The work order template construction submodule is used to map structured semantics to a standardized work order structure. The fields of the standardized work order structure include problem description, responsible department, event type, response level, time limit requirement, and warning status.

[0197] The work order status management submodule is used to track the status transition of work orders, including: "Unprocessed → Accepted → Processing Completed → User Feedback → Archived";

[0198] The API integration and dispatch submodule is used to support interaction with mainstream government platforms (such as the 12345 hotline platform and government cloud business system) through RESTful API, and to accurately dispatch work orders to the business system;

[0199] The blockchain auditing sub-module is used to record each step of the work order operation (such as transfer, processing, and feedback) on the blockchain to achieve traceability, tamper-proof, and automatic evidence storage.

[0200] The cross-departmental collaboration engine is used to handle collaborative events involving multiple responsible parties, generate sub-task assignment orders, and dynamically assign sub-work orders through a responsibility decomposition tree.

[0201] The user feedback and closed-loop optimization module in this embodiment includes:

[0202] The results push submodule is used to push the processing completion results to the user's client (App, SMS, etc.) and display the processor, processing progress and processing basis;

[0203] The satisfaction evaluation interface submodule is used to guide users to evaluate the processing effect and collect subjective ratings and text opinions;

[0204] The user behavior analysis submodule is used to build feedback tags based on user clicks, viewing paths, and frequency of follow-up questions.

[0205] The RLHF training mechanism submodule is used to introduce the "Human Feedback Reinforcement Learning (RLHF)" strategy, which uses satisfaction as a reward function to fine-tune the DeepSeek model and optimize its intent understanding, responsibility matching, and question answer generation capabilities.

[0206] The dynamic strategy optimization submodule is used to automatically update the intent classifier parameters, multimodal fusion weights, and responsibility allocation path selection rules based on feedback data.

[0207] This embodiment also includes a model and policy dynamic update module implemented through incremental learning and online update mechanisms, ensuring that the system can continuously improve and adapt to new challenges without restarting, including the following steps:

[0208] ① Through incremental data collection and processing, new data is received in real time and added to the training set;

[0209] ② The model is quickly adjusted through online learning algorithms, and the model is seamlessly updated in the background without affecting online services;

[0210] ③ By adjusting adaptive strategies, the system automatically optimizes processing strategies based on real-time data and user feedback to ensure continuous improvement in processing efficiency.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent allocation and processing of government affairs events based on a multimodal large model, characterized in that, The method is as follows: Multimodal data acquisition and preprocessing: Receive and parse modal information of text, voice and images submitted from government affairs clients, construct an initial event corpus and complete data cleaning and structuring to obtain preprocessed multimodal data; Multimodal semantic fusion and intent recognition: The DeepSeek-R1 model is used to perform multimodal semantic fusion on the preprocessed multimodal data to identify the user's core intent and event content; Event classification and responsible department matching: The structured event semantics are matched with the knowledge graph of government agencies to identify the corresponding processing unit or responsible department and assign tasks. GAT Path Reasoning: Obtain the relevance score between each department and the event through graph convolutional layers, graph attention layers and attention mechanisms, and designate the department with the highest score as the responsible department; Intelligent work order generation and cross-departmental collaboration: Automatically generate processing work orders based on the identification results, and complete responsibility assignment, collaborative tracking and lifecycle management; User feedback and closed-loop optimization: Feedback of processing results to users and collection of user feedback information for system self-optimization and iteration.

2. The intelligent allocation and processing method for government affairs events based on a multimodal large model according to claim 1, characterized in that, The multimodal data acquisition and preprocessing are detailed below: Image enhancement: Dual-domain processing is used to improve image sharpness, remove noise, and segment regions of uploaded images. Specifically, five convolutional layers plus ReLUctant are used to improve sharpness, then wavelet swap is used for x3 noise suppression, and finally, a fine segmentation map is generated through U-Net to extract scene ROIs. OCR Recognition: The Tesseract+ self-trained Chinese OCR model is used to recognize text information in images and key content fields are extracted by combining location features; Key object extraction: Based on the Swin-Transformer-Tiny pre-trained + Fine-Tune model, identify entity elements of traffic signs, government documents, and posters in the scene; Speech transcription and voice emotion feature extraction: The DeepSeek-Voice ASR system is called to transcribe speech information into text in real time. At the same time, 16-dimensional MFCC, pitch, and energy features are extracted, and a 2-layer LSTM model is used to output a voice emotion distribution probability vector. The voice emotion distribution probability vector includes speaker features and tone emotion, which are used for subsequent emotion judgment. Metadata binding: Each input record is tagged with a timestamp, user ID, latitude and longitude, and device ID, and uniformly transferred to the event corpus database to construct a structured initial corpus; the schema fields in the structured initial corpus include event_id, timestamp, user_id, modality, raw_data, geo, and preprocessed_data; Preprocessing: After the initial data collection is completed, the collected data will be cleaned and fused to ensure the consistency and accuracy of the input data; and the collection results will be standardized, specifically: text input will be uniformly encoded as UTF-8; images and audio will be deduplicated using MD5 signatures; and field integrity check rules will be set.

3. The intelligent allocation and processing method for government affairs events based on a multimodal large model according to claim 1, characterized in that, Multimodal semantic fusion and intent recognition are detailed below: Unified Semantic Encoding: Inputs from different modalities are projected onto a unified 512-dimensional semantic space, concatenated, and with positional embeddings added before being fed into a cross-modal Transformer layer (8 layers, 12 attention heads, dropout = 0.1) to ensure comprehensive analysis of data from different modalities at the same semantic level. Each modal input is processed by a corresponding Encoder: Text SeqEncoder: A Bi-LSTM model is used to process the text, combined with attention layers to capture long-range semantic information; Image Encoder: Cross-modal fusion technology using ResNet50 and SwinTransformer is used to further process image data, ensuring that image information can be effectively combined with other modal information; Speech Encoder: A 3-layer CNN + 2-layer LSTM encoding of vocal features is used to extract vocal features for subsequent sentiment analysis. Multimodal alignment training: A multimodal alignment loss function is introduced to better align modalities such as images, text, and speech, strengthening the semantic consistency of visual-text-speech representation and improving robustness and accuracy under semantic ambiguity conditions. The multimodal alignment loss function takes the following form: Among them, v i , t i a i These are the image, text, and speech encoding vectors for the same event i, respectively. Weighted Consolidation of Main Task Losses The alignment loss is calculated using the following formula: λ is initially set to 0.5 and can be dynamically adjusted; Modality Dropout is introduced during the training phase to enhance robustness. Intent Extraction: Combining historical corpora and multi-turn dialogue content, the core user intent is identified through a long-range attention mechanism. Key elements of the event are extracted using a classifier and information extractor, including event type, target audience, time of occurrence, location, and service category. Entity extraction accurately identifies the core requirements of the event, leading to subsequent event classification and responsibility matching. Specifically, the vector output from the Transformer layer is further fed into the classifier and information extractor: entity extraction is achieved using CRF+Bi-LSTM; the intent classifier is a 3-layer DNN, with softmax outputting the event type. Structured semantic output: Finally, a structured event object is generated, along with structured semantic tags.

4. The intelligent allocation and processing method for government affairs events based on a multimodal large model according to claim 1, characterized in that, The specific event classification and responsibility matching are as follows: Constructing a government knowledge graph: A government knowledge graph containing entities and edges of "item name - department function - administrative division - hierarchical relationship" is constructed, and a high-dimensional representation of the nodes of the government knowledge graph is built using a graph neural network. The features of the government knowledge graph nodes include department functions, processing timeliness, and historical event data. A hierarchical construction strategy is adopted to ensure the dynamic expansion and updating of the government knowledge graph. The government knowledge graph undergoes incremental learning monthly, maintaining its accuracy by adding data, modifying functional descriptions, or adjusting departmental relationships. Node encoding: Graph neural networks are used to encode each node in the government knowledge graph. Specifically: Node feature processing: Each node represents a government department or matter. The features of each node are initialized through vectorized text descriptions and historical data information; GNN encoding: Graph Attention Networks model is used to represent nodes in high-dimensional vector form. The vector representation of a node can reflect the function, geographical location, and historical processing capabilities of the corresponding node. Path Matching and Reasoning: The GAT model is used for path reasoning to identify the responsible department for handling events. The GAT model introduces a graph attention mechanism to calculate weighted relationships between nodes. The path reasoning algorithm finds the optimal responsible unit through multi-hop paths. Specifically: Path Reasoning: In the government knowledge graph, from the "geographical jurisdiction" of the event to the "item keywords" and then to the "responsible department," the GAT model performs multi-hop reasoning to uncover the optimal handling department. Reasoning Process: The GAT model infers the confidence of the path by focusing on the relationships between nodes in the government knowledge graph. Finally, based on the path weights, the most suitable responsible department is selected. Ranking of Responsible Entities: Multiple matching candidate units are scored and ranked, taking into account factors such as edge weights between nodes, historical processing timeliness, and user evaluations. The department with the highest overall score will become the responsible entity for the incident. Edge weights include the department's historical processing timeliness and service quality. Historical processing timeliness means that departments with fast processing speeds will be prioritized. User evaluations refer to the assessment of the corresponding department's processing capabilities based on user feedback on the corresponding department's historical processing results. Fuzzy classification processing: In cases of fuzzy expressions or polysemous words, deep semantic matching and prior event statistical models are combined to improve the accuracy of responsibility division, and ambiguity is automatically identified and eliminated by comparing historical event data.

5. The intelligent allocation and processing method for government affairs events based on a multimodal large model according to claim 1, characterized in that, GAT path reasoning is as follows: Graph Convolutional Layers: In graph convolutional layers, node features are updated based on information from their neighboring nodes; the node representation of each layer is calculated by weighted averaging of information from its neighboring nodes, and the updated node representation contains richer contextual information. Graph Attention Layer: Calculates the attention weights between nodes. These attention weights are used to focus on important nodes and edges in the graph, thereby improving the accuracy of inference. The weighted average of nodes in each layer of GAT is calculated, and multi-hop inference is performed based on the structure of the graph. Attention mechanism: By learning the relationships and semantic distances between nodes, the attention between nodes is automatically adjusted, thereby improving the accuracy of event responsibility matching.

6. The intelligent allocation and processing method for government affairs events based on a multimodal large model according to claim 1, characterized in that, The intelligent work order generation and cross-departmental collaboration are detailed below: Create work order templates: Generate standardized work order templates based on the structured semantics of events; each work order includes fields such as problem description, responsible department, event type, response level, time limit requirement, and warning status; Cross-departmental work order splitting: Handling collaborative events involving multiple responsible parties, generating sub-task assignment orders, and dynamically assigning sub-work orders through a responsibility decomposition tree; that is, when an event involves multiple departments, the work order is split to ensure that each responsible department receives the relevant part of the work order; The core of cross-departmental work order splitting lies in the reasonable decomposition of work order tasks based on the responsibility decomposition tree algorithm, ensuring cross-departmental collaboration. Specifically, the responsibility decomposition tree algorithm is used to split work orders across multiple departments and assign specific tasks, including: Task decomposition: Based on event type, relevant departmental functions, and processing time, the work order is split into multiple sub-tasks; Decomposition rules: Each sub-task is handled by one or more responsible departments, and each department is only responsible for a portion of the work within its corresponding functional scope; Upstream and downstream relationships: Sub-tasks may depend on the completion of other sub-tasks, such as a repair task depending on a preceding review task. Work order status management: Track the status transition of work orders. Work order status includes: "Unprocessed → Accepted → Processing Completed → User Feedback → Archived"; API Integration and Dispatch: Supports interaction with mainstream government platforms via RESTful API to accurately dispatch work orders to business systems; Blockchain auditing records every step of the work order's dispatch, processing, and feedback on the blockchain, enabling traceability, tamper-proofing, and automatic evidence storage.

7. The intelligent allocation and processing method for government affairs based on a multimodal large model according to any one of claims 1 to 6, characterized in that, The specific user feedback and closed-loop optimization are as follows: Results push: Push the processing completion result to the user's client and display the processor, processing progress and processing basis; Satisfaction evaluation interface: guides users to evaluate the processing effect and collects subjective ratings and text opinions; User behavior analysis: Construct feedback tags based on user clicks, viewing paths, and frequency of follow-up questions; RLHF Training Mechanism: Introducing the "Human Feedback Reinforcement Learning (RLHF)" strategy, the DeepSeek model is fine-tuned using satisfaction as the reward function to optimize its intent understanding, responsibility matching, and question-and-answer generation capabilities. Specifically: RLHF Fine-tuning Mechanism: The model's performance is optimized based on user satisfaction ratings as reward signals. The RLHF strategy dynamically adjusts model parameters to improve the accuracy of event intent, department matching, and question-and-answer generation. Reward Function Design: The RLHF fine-tuning mechanism is trained using the following reward function: R = α·Satisfaction Rating + β·Processing Timeliness + γ·Event Matching Degree; where α, β, and γ are weight coefficients, adjusted experimentally to ensure a balance between user satisfaction, timeliness, and event matching degree. Dynamic Policy Optimization: Intent classifier parameters, multimodal fusion weights, and responsibility allocation path selection rules are automatically updated based on feedback data. Dynamic Model and Policy Updates: The model and policies are automatically updated based on user feedback and real-time event data. This is achieved through incremental learning, online update mechanisms, adaptive adjustment, and seamless deployment of model updates.

8. A government affairs event intelligent allocation and processing system based on a multimodal large model, characterized in that, The system includes: The multimodal data acquisition and preprocessing module is used to receive and parse modal information of text, voice and images submitted from government affairs clients, construct an initial event corpus, and complete data cleaning and structuring. The multimodal semantic fusion and intent recognition module is used to map multimodal inputs of text, images and voice to a unified semantic representation space, and to identify the user's true needs, core intents and event context elements; The event classification and responsible department matching module is used to match structured event semantics with the knowledge graph of government agencies to identify the corresponding processing unit or responsible department; The intelligent work order generation and cross-departmental collaboration module is used to automatically generate processing work orders based on the recognition results, and complete responsibility assignment, collaborative tracking and lifecycle management; The user feedback and closed-loop optimization module is used to feed back the processing results to the user and collect their feedback information, which is then used for the system's self-optimization and iteration.

9. The intelligent allocation and processing system for government affairs based on a multimodal large model according to claim 8, characterized in that, The multimodal data acquisition and preprocessing module includes: The image enhancement submodule is used to improve image sharpness, remove noise, and segment regions of uploaded images. The OCR recognition submodule is used to recognize text information in images and extract key content fields by combining location features; The key object extraction submodule is used to identify entity elements in a scene based on an image recognition model; The speech transcription and analysis submodule is used to call the speech recognition engine to transcribe speech into text in real time, and at the same time extract speaker features and tone and emotion features for subsequent emotion judgment. The timestamp annotation and user identifier association sub-unit is used to bind metadata such as event submission time, user ID and location coordinates to each input modality information to build a preliminary event context; The multimodal semantic fusion and intent recognition module includes: The unified semantic encoding submodule is used to use DeepSeek-R1 to fuse Transformer architecture models and project different modal inputs into a shared semantic vector space. The context-aware parsing submodule is used to combine historical corpora and multi-turn dialogue content to identify the user's core intent through a long-distance attention mechanism. The semantic fusion submodule is used to introduce a multimodal alignment loss function to enhance the consistent expression of visual-text-speech semantics and improve robustness and accuracy under semantic ambiguity conditions. The event element extraction submodule is used to extract key information fields, including event type, target audience, time of occurrence, location, and service category. The structured semantic output submodule is used to build structured semantic tags.

10. The intelligent allocation and processing system for government affairs based on a multimodal large model according to claim 8 or 9, characterized in that, The event classification and responsible department matching module includes: The government knowledge graph construction submodule is used to build an information graph containing entities and edges of "item name-department function-administrative division-hierarchical relationship", which supports dynamic updates and incremental learning; The node encoding submodule is used to perform high-dimensional vectorization representation of graph nodes using graph neural networks; The path matching submodule is used to introduce multi-hop semantic path reasoning based on graph attention network to find the most confident responsible entity node in the path of "geographic jurisdiction → subject keyword → responsible department". The Responsibility Entity Ranking Submodule is used to score and rank multiple matching candidate units, taking into account factors such as edge weights between nodes, historical processing timeliness, and user evaluation. The fuzzy classification processing submodule is used to improve the accuracy of responsibility division by combining deep semantic matching and prior event statistical models in cases of fuzzy expressions or polysemous words. The intelligent work order generation and cross-departmental collaboration module includes: The work order template construction submodule is used to map structured semantics to a standardized work order structure. The fields of the standardized work order structure include problem description, responsible department, event type, response level, time limit requirement, and warning status. The work order status management submodule is used to track the status transition of work orders, including: "Unprocessed → Accepted → Processing Completed → User Feedback → Archived"; The API integration and dispatch submodule is used to support interaction with mainstream government platforms via RESTful API, and to accurately dispatch work orders to business systems. The blockchain auditing submodule is used to record each step of the work order on the blockchain, so as to achieve traceability, tamper-proof, and automatic evidence storage; A cross-departmental collaboration engine is used to handle collaborative events involving multiple responsible parties, generate sub-task assignment orders, and dynamically assign sub-work orders through a responsibility decomposition tree. The user feedback and closed-loop optimization module includes: The results push submodule is used to push the processing completion results to the user client and display the processor, processing progress and processing basis; The satisfaction evaluation interface submodule is used to guide users to evaluate the processing effect and collect subjective ratings and text opinions; The user behavior analysis submodule is used to build feedback tags based on user clicks, viewing paths, and frequency of follow-up questions. The RLHF training mechanism submodule is used to introduce the "human feedback reinforcement learning" strategy, which uses satisfaction as a reward function to fine-tune the DeepSeek model and optimize its intent understanding, responsibility matching, and question answer generation capabilities. The dynamic strategy optimization submodule is used to automatically update the intent classifier parameters, multimodal fusion weights, and responsibility allocation path selection rules based on feedback data.

Images