Government affair service automatic approval system based on multi-modal fusion and reinforcement learning

Abstract

The invention relates to the field of information automation, in particular to an automatic government affair service approval system and method based on multi-modal fusion and reinforcement learning. Multi-modal data such as government affair application texts, images and voices are collected, and after feature extraction and fusion, automatic approval is completed by means of a dynamic self-adaptive approval rule engine and a cross-department collaborative process optimization mechanism in combination with an intelligent decision-making and risk assessment system. An approval strategy is optimized through reinforcement learning, credible sharing of data is guaranteed through a block chain, and system self-optimization is achieved through online learning. Compared with a traditional examination and approval mode, the technology can improve examination and approval efficiency by 83%, improve accuracy by 17%, effectively solve the problems of difficult information processing, examination and approval rigidness and the like, and improve government affair service quality and efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of government informationization and artificial intelligence, specifically to a government service automation approval system and method based on multi-modal fusion and reinforcement learning, which is used to realize efficient and automated processing of government approval processes and improve the quality and efficiency of government services. BACKGROUND

[0002] Insufficient information processing capacity: Traditional government approval systems mainly rely on manual entry and processing of document information. In the face of a large number of application materials with various formats (such as paper documents, electronic spreadsheets, pictures, etc.), it is difficult to quickly and accurately extract key information. Although optical character recognition (OCR) technology can process text, it is not effective in processing unstructured information and mixed language content in images, resulting in incomplete information extraction and high error rates, which affects the accuracy of approval.

[0003] Rigid approval rules: The approval rules of existing systems are mostly fixed logical judgments, which cannot adapt to complex and variable government policies and actual business scenarios. When policies are adjusted or special application situations arise, the system cannot flexibly adjust the approval strategy and still requires manual intervention, which cannot achieve true automation of approval, reducing the efficiency and quality of approval.

[0004] Low process coordination efficiency: Government approval usually involves the coordinated work of multiple departments, but the systems of different departments are independent of each other, making data sharing difficult and information transmission delayed and error-prone. For example, a project approval may need to go through planning, environmental protection, construction, and other departments, and the exchange of data between departments relies on manual transmission and repeated entry, resulting in a long approval period and high cost for businesses and the public.

[0005] Lack of intelligent decision support: Traditional approval systems can only give approval results based on pre-set rules and cannot provide decision-making basis and risk assessment for approval personnel. In the face of complex applications, it is difficult for approval personnel to quickly judge potential risks, increasing the difficulty of decision-making and the risk of approval, and is not conducive to the standardization and standardization of government services. SUMMARY

[0006] The purpose of the present application is to provide a government service automation approval system based on multi-modal fusion and reinforcement learning, comprising:

[0007] A multi-modal data acquisition module for collecting text, image, voice and other multi-type data in government applications;

[0008] A multi-modal information fusion processing module containing a CNN image feature extraction unit, an RNN text feature extraction unit, an MFCC voice feature extraction unit, and a feature fusion network for feature extraction and fusion of multi-modal data to generate a unified feature vector;

[0009] Dynamic adaptive approval rule engine based on knowledge graph and NLP technology, combined with reinforcement learning algorithm, to realize dynamic updating and strategy optimization of approval rules;

[0010] Cross-departmental collaborative process optimization mechanism, including unified data exchange interface, intelligent process scheduling module and blockchain data recording module, for realizing cross-departmental data sharing and approval process optimization;

[0011] Intelligent decision-making and risk assessment system, using machine learning algorithm to establish risk assessment model, and providing decision basis and risk report through explanatory artificial intelligence technology;

[0012] System self-optimization and continuous learning framework for collecting data for online learning, realizing system performance evaluation and optimization.

[0013] Further, the feature fusion network adopts weighted summation or deep neural network fusion method to fuse image, text and speech features into unified feature vector.

[0014] Further, the dynamic adaptive approval rule engine analyzes policy documents through NLP technology, automatically updates knowledge graph, and adjusts approval rules according to application information and historical approval results using reinforcement learning algorithm.

[0015] Further, the cross-departmental collaborative process optimization mechanism uses blockchain technology to record approval data, ensuring data authenticity, non-tamperability and traceability.

[0016] Further, the intelligent decision-making and risk assessment system automatically triggers different approval processes according to the results of the risk assessment model.

[0017] Further, the system self-optimization and continuous learning framework uses online learning algorithm to update system model parameters in real time.

[0018] Further, the government service automation approval method includes the following steps:

[0019] Collecting multi-modal data such as text, image, speech in government application and preprocessing;

[0020] Using CNN, RNN and MFCC to extract image, text and speech features respectively, and generating unified feature vector through feature fusion network;

[0021] Input the feature vector into the dynamic adaptive approval rule engine, generate the approval strategy based on the knowledge graph and reinforcement learning algorithm;

[0022] Planning the approval process path through the cross-departmental collaborative process optimization mechanism, realizing the data sharing and collaborative approval among departments;

[0023] Risk assessment is performed by using the intelligent decision-making and risk assessment system to provide a basis for decision-making and trigger a corresponding approval process.

[0024] Data is collected for online learning, and the system is evaluated and optimized for performance.

[0025] The present application has the following advantages:

[0026] (1) Multi-modal information fusion processing technology Traditional government affairs system has limited processing capacity for multi-source heterogeneous information, and it is difficult to achieve efficient integration. The present application proposes a multi-modal information fusion processing technology, constructs a multi-modal data acquisition module, and collects multi-type data such as text, image, and voice in government affairs application. Convolutional neural network (CNN) is used to extract features from image data, recurrent neural network (RNN) is used to process text sequence information, and mel frequency cepstral coefficient (MFCC) is used to extract voice features. The extracted multi-modal features are deeply fused through a feature fusion network to form a unified feature vector, providing a comprehensive and accurate information base for subsequent approval.

[0027] (2) Dynamic adaptive approval rule engine

[0028] In view of the problem of rigid existing approval rules, the present application designs a dynamic adaptive approval rule engine. Based on knowledge graph technology, the engine constructs a knowledge graph from government policies, regulations, historical approval cases, and other knowledge. By using natural language processing (NLP) technology to analyze newly introduced policy documents, the knowledge graph is automatically updated. During the approval process, relevant rules and cases are matched from the knowledge graph according to the application information, and reinforcement learning algorithm is used to dynamically adjust and optimize the rules. When encountering special applications, the system can automatically generate new approval strategies based on historical cases and current situations, so that the approval rules can adapt to complex and variable business scenarios.

[0029] (3) Cross-departmental collaborative process optimization mechanism

[0030] To solve the problem of low efficiency of government affairs approval process collaboration, the present application establishes a cross-departmental collaborative process optimization mechanism. A unified data exchange standard and interface are designed to break down the data barriers between departments. An intelligent process scheduling module is constructed to automatically plan the optimal approval process path based on application information and business load of each department. Blockchain technology is used to record key data in the approval process to ensure data authenticity, non-tamperability, and traceability. When the approval process is transferred between departments, the system automatically pushes the approval tasks and related information to realize efficient collaboration between departments and shorten the approval cycle.

[0031] (4) Intelligent decision-making and risk assessment system

[0032] The intelligent decision-making and risk assessment system developed by the present application makes up for the lack of intelligent decision support in traditional systems. By training historical approval data through machine learning algorithms, a risk assessment model is established to predict potential risks of the application project. Using explainable artificial intelligence technology, detailed decision-making basis and risk analysis reports are provided for the approval results. The system can also automatically trigger different approval processes according to the risk level, such as high-risk projects entering the manual review link and low-risk projects speeding up the approval process, assisting approval personnel in making scientific decisions and reducing approval risks.

[0033] (Five) System self-optimization and continuous learning framework

[0034] To ensure the long-term efficient operation of the system, the present application proposes a system self-optimization and continuous learning framework. Collect user feedback, approval results and newly generated data for continuous learning and optimization of the system. Online learning algorithms are used to update model parameters in real time, making the system adapt to changing business needs and data distribution. The performance of the system is evaluated regularly, and the system architecture and algorithms are optimized according to the evaluation results to improve the stability and intelligence level of the system. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 Overall process flow diagram. DETAILED DESCRIPTION

[0036] The embodiments of the present application will be further described in detail below in conjunction with the drawings and examples. The following examples are used to illustrate the present application, but cannot be used to limit the scope of the present application.

[0037] Example 1

[0038] (I) Equipment and system preparation

[0039] Hardware equipment: Deploy high-performance server clusters for running various algorithms and storing data of the system; configure high-resolution scanners, audio recording equipment and other data acquisition hardware to ensure high-quality acquisition of multi-modal data.

[0040] Software system: Build a government service platform based on cloud computing, install multi-modal information processing, approval rule engine, intelligent decision-making and other software modules; deploy blockchain nodes to realize trusted sharing of cross-department data; integrate mainstream artificial intelligence development frameworks such as TensorFlow, PyTorch, etc. to support algorithm development and running.

[0041] (II) Process steps

[0042] Multi-modal data collection and preprocessing: Applicants submit application materials through the government service platform. The system uses scanners, voice input devices, etc. to collect text, image, voice and other data. Preprocessing is performed on the collected data, such as image denoising, text normalization, voice noise reduction, etc. to prepare for subsequent processing.

[0043] Multi-modal information fusion processing: The preprocessed data is input into the corresponding feature extraction network. CNN extracts key information from images (such as text and seals in ID photos), RNN processes text data to extract semantic features, and MFCC extracts acoustic features from voice. Then these features are input into the feature fusion network, using weighted summation or deep neural network fusion to generate a unified feature vector.

[0044] Dynamic approval rule matching and optimization: The fused feature vector is input into the dynamic adaptive approval rule engine. The system matches relevant approval rules and historical cases from the knowledge graph. Using reinforcement learning algorithms, the system adjusts and optimizes the matched rules according to the current application situation and historical approval results, generating an approval strategy suitable for the current application.

[0045] Cross-departmental collaborative approval process execution: The intelligent process scheduling module plans the approval process path based on the approval strategy and the business load of each department. The system automatically pushes the approval task and related information to the corresponding department. Each department receives the task and performs the approval operation according to the specified process. Data in the approval process is recorded and shared through blockchain, ensuring information transparency and traceability.

[0046] Intelligent decision-making and risk assessment: The intelligent decision-making and risk assessment system uses risk assessment models to predict the risk level of the application project based on the approval strategy and application information. It also generates detailed decision-making basis and risk analysis reports to assist approval personnel in making decisions. For high-risk projects, the system automatically triggers manual review processes; for low-risk projects, it speeds up the approval process to achieve quick settlement.

[0047] System self-optimization and continuous learning: The system collects user feedback (such as satisfaction with the approval results, suggestions, etc.), approval results, and newly generated data, and updates the model using online learning algorithms. The system's performance indicators (such as approval accuracy and processing efficiency) are evaluated regularly, and the system's algorithms, parameters, and architecture are optimized based on the evaluation results to continuously improve the system's service quality.

[0048] (Three) Specific case analysis

[0049] Case 1: Under the traditional government approval mode, an enterprise needs to go to multiple departments such as planning, housing and construction, and environmental protection to submit materials for construction engineering construction license, and the approval period is as long as 30 working days, and the enterprise needs to go back and forth to correct the materials due to incomplete materials. After using the government service automatic approval system of the application, the enterprise submits multi-modal application materials through the platform at one time, the system automatically extracts key information and performs fusion processing. The dynamic self-adaptive approval rule engine quickly generates an approval strategy according to policies and regulations and historical cases, and the intelligent process scheduling module plans an optimal approval path to realize efficient collaboration between departments. Finally, the construction license approval of the enterprise is completed in only 5 working days, the approval efficiency is improved by 83%, and the business cost of the enterprise is greatly reduced.

[0050] Case 2: When handling personal entrepreneurship loan approval, the traditional system is difficult to comprehensively evaluate the complex information such as the business plan and credit status of the applicant, and the accuracy of the approval result is low. The system comprehensively analyzes the information such as the text plan, financial statement image and application voice submitted by the applicant through multi-modal information fusion processing. The intelligent decision and risk assessment system uses the trained model to accurately evaluate the repayment ability and risk of the applicant, and provides a detailed analysis report. According to statistics, after using the system, the accuracy of personal entrepreneurship loan approval is improved from 75% to 92%, effectively reducing the financial risk, and improving the convenience of entrepreneurs to obtain loans.

[0051] Embodiments of the application are given for the purpose of illustration and description, and are not intended to be exhaustive or to limit the application to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are chosen and described in order to best explain the principles of the application and its practical application, and to enable others skilled in the art to understand the application for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A government service automated approval system based on multimodal fusion and reinforcement learning, characterized by: include:

1. Multimodal data collection module, used to collect multiple types of data such as text, images, and voice in government applications; 2. Multimodal information fusion processing module, including CNN image feature extraction unit, RNN text feature extraction unit, MFCC speech feature extraction unit and feature fusion network, used to extract and fuse features of multimodal data to generate a unified feature vector; 3. Dynamic adaptive approval rule engine, based on knowledge graph and NLP technology, combined with reinforcement learning algorithm, to achieve dynamic update of approval rules and strategy optimization; 4. Cross-departmental collaborative process optimization mechanism, including a unified data exchange interface, an intelligent process scheduling module, and a blockchain data recording module, to achieve cross-departmental data sharing and approval process optimization; 5. Intelligent decision-making and risk assessment system, which uses machine learning algorithms to establish risk assessment models and provides decision-making basis and risk reports through explanatory artificial intelligence technology; 6. System self-optimization and continuous learning framework, used to collect data for online learning to achieve system performance evaluation and optimization.

2. The system according to claim 1, wherein: The feature fusion network adopts weighted summation or deep neural network fusion to fuse image, text and speech features into a unified feature vector.

3. The system according to claim 1, wherein: The dynamic adaptive approval rule engine uses NLP technology to parse policy documents, automatically updates the knowledge graph, and uses reinforcement learning algorithms to adjust approval rules based on application information and historical approval results.

4. The system according to claim 1, wherein: The cross-departmental collaborative process optimization mechanism uses blockchain technology to record approval data to ensure the authenticity, non-tamperability and traceability of the data.

5. The system according to claim 1, wherein: The intelligent decision-making and risk assessment system automatically triggers different approval processes based on the results of the risk assessment model.

6. The system according to claim 1, wherein: The system self-optimization and continuous learning framework adopts an online learning algorithm to update system model parameters in real time.

7. A method for automated approval of government services based on multimodal fusion and reinforcement learning, characterized in that: The following steps are involved:

1. Collect and pre-process multimodal data such as text, images, and voice from government applications; 2. Use CNN, RNN and MFCC to extract image, text and speech features respectively, and generate a unified feature vector through the feature fusion network; 3. Input the feature vector into the dynamic adaptive approval rule engine, and generate the approval strategy based on the knowledge graph and reinforcement learning algorithm; 4. Plan the approval process path through the cross-departmental collaborative process optimization mechanism to achieve data sharing and collaborative approval among departments; 5. Use intelligent decision-making and risk assessment systems to conduct risk assessments, provide decision-making basis, and trigger corresponding approval processes; 6. Collect data for online learning and perform performance evaluation and optimization of the system.

8. The method according to claim 7, characterized in that In the feature fusion step, the weighted summation formula is used Perform feature fusion, where V is the fused feature vector, w i is the weight of each modal feature, F i is the feature vector extracted from each mode.

9. The method according to claim 7, characterized in that In the step of generating the approval strategy by the dynamic adaptive approval rule engine, the Q-learning algorithm of reinforcement learning is used to calculate the approval strategy through the formula Q(s,a)←Q(s,a)+α·[r+γ·max a′ Q(s′,a′)-Q(s,a)] updates the approval rule, where Q(s,a) is the Q value of taking action a in state s, α is the learning rate, r is the reward value, and γ is the discount factor.

10. The method according to claim 7, characterized in that In the cross-department collaborative approval process execution step, the intelligent process scheduling module is based on the business load L of each department. i and the application urgency E, through the formula Calculate the processing priority of each department, where P i is the priority of department i.

Images