Multi-modal data fusion enterprise benefit policy declaration automatic processing system

Abstract

The invention discloses a multi-modal data fusion enterprise benefit policy declaration automatic processing system, which relates to the field of enterprise benefit policy declaration and comprises a data acquisition module, a preprocessing module, a fusion module, a policy matching module, a material generation module, a declaration process automation module, an intelligent feedback and optimization module, a user interaction module, a policy recommendation optimization module and a data quality evaluation module. The method comprises the following steps: acquiring enterprise internal and external data marking reliability in a multi-source manner; cleaning and analyzing normalized data by adopting an innovative algorithm; fusing data by combining CNN, RNN and a self-attention mechanism; generating and auditing materials based on a knowledge graph and deep learning matching policy and a template and NLG; and an immersive interface is created to support various interactions. According to the invention, multi-source data is collected efficiently, enterprise benefit policies are matched accurately, declaration materials are generated automatically, enterprise declaration efficiency and success rate are improved, government auditing process is optimized, data security is enhanced, policy accurate landing is promoted, and government and enterprise win-win is assisted.

Description

Technical Field

[0001] The present invention relates to the technical field of enterprise-friendly policy declaration, and in particular to an automated processing system for enterprise-friendly policy declaration with multimodal data fusion. Background Art

[0002] In the current economic environment, the government has introduced a wide variety of preferential policies to promote business development, covering tax incentives, financial subsidies, project support, and more. However, businesses face numerous challenges when applying for these policies. The traditional application process for preferential policies relies on manual processes, requiring businesses to expend considerable time and effort collecting and organizing various application materials. Application requirements vary across policies, and the format and content of these materials are complex and cumbersome, making it difficult for businesses to accurately grasp them. This leads to inefficient applications and is prone to issues such as incomplete materials and formatting errors, which in turn impacts the success rate of applications.

[0003] With the advent of the digital age, while some companies have begun utilizing information technology to assist with applications, these data sources are limited, often relying solely on internal financial and business data, which fails to fully reflect the company's actual situation. Furthermore, the evaluation criteria for preferential policies for businesses are increasingly diverse, focusing not only on economic indicators but also on multiple dimensions such as innovation capabilities, social responsibility, and market influence. This single data source makes it difficult for companies to accurately identify appropriate policies during the policy matching process, resulting in wasted policy resources and missed development opportunities.

[0004] Furthermore, there is a lack of efficient information exchange channels between businesses and the government. During the application process, businesses' questions about policy details are not answered promptly, and the application progress is difficult to track in real time. When reviewing application materials, government departments often need to communicate and verify with businesses repeatedly due to incomplete or inaccurate information. This further lengthens the application cycle, increases administrative costs, and increases the burden on businesses. Furthermore, the existing application system lacks intelligent feedback and optimization mechanisms, making it impossible to continuously improve the application process and policy matching methods based on application results and business needs, making it difficult to adapt to the ever-changing policy environment and business development needs. Therefore, the development of an efficient, intelligent, multimodal data-fusion automated processing system for enterprise-friendly policy applications is of urgent practical significance. Summary of the Invention

[0005] The present invention proposes an automated processing system for enterprise-friendly policy declaration based on multimodal data fusion to solve the problems mentioned in the above-mentioned prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a multimodal data fusion enterprise policy declaration automatic processing system, comprising:

[0007] Multimodal data collection module: adopts a combination of active and passive collection mechanisms to obtain data from within the enterprise, uses web crawlers to capture public information, uses digital watermarks to mark data, and uses a weighted coefficient method to assign weights to data from different sources;

[0008] Data preprocessing module: proposes data cleaning and normalization strategies, uses a combination of rules and machine learning algorithms to detect outliers for structured data, processes unstructured data through semantic analysis, and uses a dynamic range adjustment formula for feature normalization;

[0009] Multimodal data fusion module: Design a multimodal neural network architecture, combine CNN, RNN and self-attention mechanism, and realize data fusion through multi-head attention calculation formula;

[0010] Policy matching module: Builds a policy matching engine based on knowledge graph and deep learning, uses graph embedding and twin networks to calculate similarity, introduces a dynamic update mechanism, and adjusts the matching strategy according to the policy update time;

[0011] Application materials generation module: This module uses a template-driven and natural language generation (NLG) approach to generate text based on policies and corporate data, uses a perplexity formula to assess text quality, and checks and corrects text according to grammatical rules and policy requirements.

[0012] Declaration process automation module: Using blockchain technology, triggering the declaration process through smart contracts, connecting with government systems using API gateway technology, and calculating the data transmission success rate through formulas;

[0013] Intelligent feedback and optimization module: Utilizes reinforcement learning algorithms to optimize the system, uses Q-learning algorithms to update Q-values ​​through feedback signals from application results, analyzes user operation behaviors, and explores user needs;

[0014] User interaction module: Create an immersive interactive interface, integrate VR and AR technologies, support voice interaction and gesture recognition, calculate the accuracy of voice command recognition through formulas, and dynamically adjust the interface layout and interaction methods based on user habits and preferences.

[0015] Furthermore, it also includes:

[0016] Policy recommendation optimization module: Use transfer learning technology to transfer model parameters, the new model parameters θ new Denoted as θ new =(1-β)θ old +βθ train ,θ old is the old model parameter, θ train is the parameter for new task training, and β is the migration coefficient. Combining enterprise development strategy and market trend forecast, the Bayesian network model is used to evaluate the potential benefits E of the policy. The formula is: P(S i ) is the policy in state S i The probability of occurrence under i ) is the profit in this state.

[0017] Furthermore, it also includes:

[0018] Multimodal data quality assessment module: Introducing the fuzzy comprehensive evaluation method, assuming that the data quality assessment index set U = {u1,u2,…,u m}, comment set V = {v1,v2,…,v n}, fuzzy relationship matrix R=(r ij ) m×n , weight vector A=(a1,a2,…,a m ), the comprehensive evaluation result B=A○R, ○ is the fuzzy synthesis operator, and the maximum-minimum synthesis method is used. j = 1, 2, ..., n; the data is graded according to the scores, with a focus on marking and repairing low-quality data.

[0019] Furthermore, the weighted coefficient method is used to allocate the importance of data from different sources in the multimodal data acquisition module. i Weight w i The calculation formula is f i For data D i Credibility factor;

[0020] Increase the data collection of enterprise IoT devices, obtain equipment operation status information by integrating with production equipment and sensors, and use federated learning technology. The local model update formula is: is the model parameter of the i-th enterprise in the t-th round, α is the learning rate, L is the loss function, D i is the local data of the i-th enterprise, and the global model parameters w i is the weight of the i-th enterprise, and k is the number of enterprises.

[0021] Furthermore, the structured data in the data preprocessing module is combined with rule-based and machine learning algorithms to detect data anomalies through the local outlier factor LOF formula, which is: p is the data point, N k (p) is the k-neighborhood of p, lrd is the local reachability density; unstructured data uses feature normalization Dynamic range adjustment, is the dynamic mean of the data, is the dynamic standard deviation, m is the number of data samples;

[0022] Combine semi-supervised learning and active learning to label unstructured data. Semi-supervised learning uses labeled data D label and unlabeled data D unlabel Training model, model loss function L = L label (D label )+λL unlabel (D unlabel ), L label and L unlabel are the loss functions for labeled and unlabeled data respectively, and λ is the balance coefficient; active learning selects the most valuable data annotation according to the uncertainty of the model, and the uncertainty U(x) is calculated by the entropy formula Calculate, P(y i |x) is the sample x belonging to category y i The probability of c is the number of categories; the knowledge distillation technology is introduced to transfer the knowledge of large pre-trained language models to lightweight models, and the distillation loss function z s and z t are the outputs of the student model and the teacher model respectively, T is the temperature parameter, and KL is the KL divergence.

[0023] Furthermore, the multi-head attention formula in the multimodal data fusion module is MultiHead(Q, K, V) = Concat(head1, ..., head h )W O ,in Q, K, and V are query, key, and value matrices respectively, d k is the dimension of the key vector, and W O is the learning weight matrix;

[0024] Construct a cross-modal attention network with the attention weight w between different modalities m and n mn The calculation formula is s mn is the similarity score between the features of modality m and n, calculated by dot product f m and f n are the feature vectors of modalities m and n respectively; the adversarial training mechanism is introduced, and the loss functions of the generator G and the discriminator D are L G = -log(D(G(z))) and L D = -log(D(x))-log(1-D(G(z))), where x is the real data and z is the random noise.

[0025] Furthermore, the TransE model is used in the policy matching module. The embedding vectors of the relationship r corresponding to the entities h and t satisfy h+r≈t, and the similarity sim adopts the cosine similarity formula A and B are vector representations of enterprise data and policy conditions, respectively;

[0026] Using meta-learning technology, the meta-update formula is θ is the meta-model parameter, β is the meta-learning rate, For task T i The loss function is α is the inner loop learning rate, S i For task T i support set.

[0027] Furthermore, the quality assessment of text generation in the application materials generation module uses the perplexity formula W=w1,…,w N is the generated text sequence, P(w i |w1,…,w i-1 ) is the probability of predicting the i-th word based on the language model;

[0028] The application materials are generated by combining the generative adversarial network (GAN) and the variational autoencoder (VAE). GAN generates diversified texts, while VAE controls the semantic constraints and quality of the generated texts. The loss function of VAE is L VAE =L recon +λL KL , L recon is the reconstruction loss, L KL is the KL divergence loss; the knowledge graph enhanced text generation technology is introduced, and the influence factor I of the entity e and relationship r in the knowledge graph on text generation er By formula Calculate, count(e,r) is the number of times entity e and relation r appear in the knowledge graph.

[0029] Furthermore, the voice command recognition accuracy Acc in the user interaction module is calculated by the formula Calculation, R correct is the number of correctly recognized instructions, R total is the total number of instructions;

[0030] Add sentiment analysis function, identify the emotional state by analyzing the language expression and operation behavior of the user's interaction with the system, and use the sentiment classification model. The model outputs the probability of category y z y is the score of category y; adjust the interaction strategy according to the sentiment analysis result S. If S is dissatisfied, the probability P of the system automatically providing a solution to the problem solution By formula Calculation, time unsolved Time is the time when the problem is not solved. total is the total interaction time.

[0031] Furthermore, the smart contract execution conditions in the declaration process automation module C i For the i-th condition, the data transmission success rate S is calculated by the formula Calculation, M success is the amount of data successfully transmitted, M total is the total amount of data transmitted;

[0032] Introducing quantum encryption technology, quantum key distribution uses the BB84 protocol, and the key generation rate R is calculated by the formula Calculation, p is the probability of single photon emission, η is the detector efficiency; the application process is managed by smart contracts, and the contract execution time T stepi is the execution time of the i-th step.

[0033] Compared with the existing technology, the beneficial effects of the present invention are:

[0034] For businesses, this greatly improves application efficiency. The system automatically collects multimodal data, eliminating the need for businesses to manually collect and organize large amounts of information, saving significant time and labor costs. The precise policy matching function, based on comprehensive enterprise data, can quickly screen for preferential policies that meet the company's actual situation, preventing companies from blindly applying and significantly increasing the success rate of policy applications. Through an immersive user interface, businesses can quickly and easily query policy information, track application progress, and receive timely intelligent feedback and operational guidance, enhancing the user experience.

[0035] From a government perspective, the system optimizes the review process. The automated application process reduces manual intervention, lowers review errors, and improves review efficiency. The system provides complete and accurate enterprise data, helping the government gain a more comprehensive understanding of enterprise situations and make more informed review decisions. The intelligent feedback and optimization module continuously optimizes system functionality based on application results and enterprise feedback, better adapting to policy adjustments and evolving enterprise needs, and ultimately enhancing the government's ability and level of service to enterprises.

[0036] Furthermore, the system's data security and privacy protection measures ensure the safety of corporate data, alleviating concerns about data leaks. Multimodal data integration makes policy application and review more scientific and comprehensive, promoting the precise implementation of policies that benefit enterprises. This helps drive enterprise innovation and development, stimulates market vitality, and creates a win-win situation for both government and enterprises, providing strong support for sustainable economic and social development. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a schematic block diagram of an automated processing system for enterprise benefit policy declaration based on multimodal data fusion proposed by the present invention;

[0038] Figure 2 A diagram comparing the time required to prepare application materials;

[0039] Figure 3 This is a schematic diagram of the policy matching accuracy trend;

[0040] Figure 4 This is a diagram showing the success rate of enterprise applications. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0043] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. The present invention will be further described in detail below with reference to the accompanying drawings.

[0044] Reference Figures 1 to 4:A multi-modal data fusion enterprise policy declaration automatic processing system, including:

[0045] Multimodal data acquisition module: adopts a collection mechanism that combines active and passive methods. On the one hand, structured data such as financial statements and sales data are obtained from the company's internal information systems such as ERP and CRM through interfaces; on the other hand, web crawler technology is used to capture the company's market reputation, industry trends and other unstructured information from public channels such as industry forums and social media according to preset rules and frequencies. For offline paper documents, such as contracts and invoices, image recognition and OCR technology are used for scanning and information extraction. To ensure the reliability of the data, a digital watermark is added to the collected data. The generation of digital watermarks is based on information such as the source of the data, the time of collection, and the characteristics of the data, and a unique watermark identifier is generated through a specific encryption algorithm. In the subsequent use and tracing of the data, the authenticity and integrity of the data can be verified through a decryption algorithm. For data from different sources, weights are assigned according to their credibility factors. Credibility factor f i The data is determined by comprehensively considering the authority of the data source, update frequency, data accuracy and other factors. i The weight w i The calculation formula is For example, the data in a company's internal financial system is more credible and has a relatively larger weight; while some data on social media is less credible and has a correspondingly smaller weight.

[0046] Data preprocessing module: It uses an outlier detection algorithm based on a combination of rules and machine learning. First, it preliminarily screens out possible outliers based on preset rules, such as the value range and logical relationship of the data. Then, it uses the local outlier factor (LOF) formula to Further determine whether the data point p is an outlier. For detected outliers, mean filling, interpolation and other methods are used to correct them according to the distribution of the data. A semantic understanding model is introduced to perform deep semantic analysis on text data. This model is based on pre-trained language models such as BERT to extract features and represent semantics of text. After completing the semantic analysis, operations such as word segmentation, part-of-speech tagging, and named entity recognition are performed to convert the text data into a format that is convenient for subsequent analysis. The dynamic range adjustment formula is used. in is the dynamic mean of the data, is the dynamic standard deviation, and m is the number of data samples. In this way, data from different modalities can be compared.

[0047] Multimodal data fusion module: Deep fusion multimodal neural network architecture, combining convolutional neural network (CNN), recurrent neural network (RNN) and self-attention mechanism. For image data and structured data, CNN is used to extract their spatial features. CNN automatically learns local features and patterns in the data through operations such as convolution layers and pooling layers. For sequence data, such as time series business data, text data, etc., RNN and its variants (such as LSTM, GRU) are used to process their temporal dependencies. RNN can capture long-term dependency information in sequence data. The self-attention mechanism is introduced to dynamically assign weights according to the importance of the data. The multi-head attention formula is MultiHead(Q, K, V) = Concat(head1, ..., head h )W O ,in Q, K, and V are query, key, and value matrices respectively, d k is the dimension of the key vector, and W O is a learnable weight matrix. In this way, the intrinsic correlation between multimodal data can be captured more comprehensively and accurately.

[0048] Policy matching module: Build a policy matching engine based on knowledge graph and deep learning, structure the terms, applicable conditions, characteristics of related enterprises and other information of the enterprise-friendly policies, and build a knowledge graph. Using the TransE model, the embedding vectors of entities h and t corresponding to the relationship r satisfy h+r≈t. The information in the knowledge graph is converted into a low-dimensional vector through graph embedding technology to facilitate subsequent calculations and matching. The deep learning model takes the multimodal fused data and policy vector as input, and uses the twin network structure to calculate the similarity score between enterprise data and policy. The similarity sim uses the cosine similarity formula Where A and B are vector representations of enterprise data and policy conditions respectively. Based on the similarity score, the enterprise-friendly policies that meet the enterprise situation are screened out and sorted according to the score. The system introduces a policy dynamic update mechanism to track policy changes in real time. According to the policy update time T new and last matching time T old , if T new >T old , the matching strategy is automatically adjusted and the matching degree between the enterprise and the policy is recalculated.

[0049] Application materials generation module: The application materials generation method based on template drive and natural language generation (NLG) is adopted. The system predefines application materials templates for various policy types, including forms, documents and other formats. According to the matched policy, relevant information is automatically extracted from the multimodal data and filled into the template. For example, for financial data, it is directly extracted from the database and organized into a table format that meets policy requirements. For parts that require text description, such as corporate development status, project implementation plans, etc., the NLG model combines the actual situation of the enterprise and policy requirements to generate a logically clear and linguistically standardized application text. For the quality assessment of text generation, the perplexity formula is used where W=w1,…,w N is the generated text sequence, P(w i |w1,…,w i-1 ) predicts the probability of the i-th word based on the language model. The system has an intelligent review function that checks and corrects generated materials according to policy requirements and grammatical rules. For example, it checks whether the data in the application materials is complete, the logic is reasonable, and the grammar is correct.

[0050] Declaration process automation module: Blockchain technology is used to achieve transparency and immutability of the declaration process. All stages of the declaration process (such as material submission, review status update, etc.) are recorded on the blockchain to form an immutable audit trail. Each transaction record contains information such as the declaration time, the content of the declaration materials, and the review opinions, ensuring the openness and transparency of the declaration process. Each step of the declaration process is automatically triggered by the smart contract. The execution condition C of the smart contract can be expressed as Among them C i is the i-th condition, such as completeness of materials, time of application, etc. For example, when the application materials are complete and submitted within the prescribed time, the smart contract automatically sends the application materials to the application system of the relevant government department. The connection with the government application system adopts API gateway technology to achieve secure and efficient data transmission. The data transmission success rate S can be calculated by the formula Calculate, where M success is the amount of data successfully transferred,

[0051] M total is the total amount of data transferred.

[0052] Intelligent feedback and optimization module: Utilizes reinforcement learning algorithm to continuously optimize the system, uses the application result (success or failure) as the feedback signal, adopts Q-learning algorithm, and the update formula is Q(s,a)=Q(s,a)+α[r+γmax a′Q(s′,a′) - Q(s,a)], where α is the learning rate, r is the reward value, γ is the discount factor, and s′ is the next state. By continuously interacting with the environment, the system learns the optimal system parameters and policies. The system also analyzes user behavior to uncover user needs. For example, it analyzes the frequency of user policy queries and preferred policy types, providing a basis for system functionality upgrades.

[0053] User interaction module: Create an immersive user interaction interface, combining virtual reality (VR) and augmented reality (AR) technology. Users can enter the virtual policy declaration environment through VR devices and intuitively understand the declaration process and policy details; through AR technology, users can use mobile phones or tablets to view enterprise-related policy information and declaration progress in real scenes. The system supports voice interaction and gesture recognition to improve the convenience of user operation and interactive experience. The recognition accuracy of voice commands Acc can be calculated by the formula Calculate, where R correct is the number of correctly recognized instructions, R total is the total number of commands. At the same time, the interface layout and interaction method are dynamically adjusted according to the user's operating habits and preferences.

[0054] The present invention also includes the following modules:

[0055] Policy recommendation optimization module: Using transfer learning technology, the model trained on existing policy data is transferred to the newly issued policy recommendation task. The parameters of the new model are θ new It can be expressed as θ new =(1-β)θ old +βθ train , where θ old is the old model parameter, θ train is the parameter for training new tasks, and β is the migration coefficient. Combining the enterprise's development strategy and market trend forecast, the Bayesian network model is used to evaluate the potential benefits E of the policy. The formula is: Where P(S i ) is the policy in state S i The probability of occurrence under i ) is the income under this state. Based on the evaluation results, more forward-looking and targeted policy recommendations are provided to enterprises.

[0056] The present invention also includes the following modules:

[0057] Multimodal data quality assessment module: A multimodal data quality assessment method based on fuzzy comprehensive evaluation is introduced. Assume that the data quality assessment index set U = {u1,u2,…,u m}, including multiple dimensions such as data accuracy, completeness, consistency, and timeliness; the comment set V = {v1, v2, ..., v n}, such as "excellent", "good", "medium", "poor", etc. Through expert evaluation or data statistical analysis, the fuzzy relationship matrix R = (r ij ) m×n , where r ij Indicates the index u i Belong to comments v j The analytic hierarchy process (AHP) is used to determine the weight vector A of each evaluation index = (a1, a2, ..., a m ). The comprehensive evaluation result B=A○R, where ○ is the fuzzy synthesis operator, and the maximum-minimum synthesis method is used. j = 1, 2, ..., n. Based on the scores, the system automatically classifies the data and focuses on marking and repairing low-quality data.

[0058] In this invention, the multimodal data acquisition module increases the collection of enterprise Internet of Things (IoT) device data. By integrating with production equipment and sensors, it can obtain information such as the equipment operation status in real time. Using federated learning technology, the local model update formula is in is the model parameter of the i-th enterprise in the t-th round, α is the learning rate, L is the loss function, D i is the local data of the i-th enterprise. Global model parameters where w i is the weight of the i-th enterprise, and k is the number of enterprises.

[0059] In the present invention, the data preprocessing module uses a method combining semi-supervised learning and active learning to label unstructured data. label and a large amount of unlabeled data D unlabel Training model, model loss function L = L label (D label )+λL unlabel (D unlabel ), where L label and L unlabel are the loss functions for labeled and unlabeled data respectively, and λ is the balance coefficient. Active learning selects the most valuable data annotation based on the model uncertainty. The uncertainty U(x) can be expressed by the entropy formula Calculate, where P(y i |x) is the sample x belonging to category y i The probability of c is the number of categories. The knowledge distillation technology is introduced to transfer the knowledge of large pre-trained language models to lightweight models. The distillation loss function where z s and zt are the outputs of the student model and the teacher model respectively, T is the temperature parameter, and KL is the KL divergence.

[0060] In this invention, the multimodal data fusion module proposes an enhanced mechanism for cross-modal feature fusion. A cross-modal attention network is constructed, and the attention weights w between different modalities m and n are mn The calculation formula is where s mn is the similarity score between the features of modality m and n, calculated by dot product f m and f n are the feature vectors of modalities m and n, respectively. Introducing the adversarial training mechanism, the loss functions of the generator G and the discriminator D are LG = -log(D(G(z))) and L D = -log(D(x))-log(1-D(G(z))), where x is the real data and z is the random noise.

[0061] In this invention, the policy matching module uses meta-learning technology. The meta-learning model uses the MAML algorithm, and the meta-update formula is θ is the meta-model parameter, β is the meta-learning rate, For task T i The loss function is α is the inner loop learning rate, S i For task T i Combined with the expert system, expert knowledge is expressed in the form of rules, such as IF condition C THEN conclusion R, to assist in judging the matching results.

[0062] In the present invention, the application materials generation module uses a method combining generative adversarial networks (GAN) and variational autoencoders (VAE) to generate application materials. GAN generates diversified texts, and VAE imposes semantic constraints and quality control on the generated texts. The loss function L of VAE is VAE =L recon +λL KL , where L recon is the reconstruction loss, L KL is the KL divergence loss, which measures the difference between the potential variable distribution and the prior distribution. Introducing the knowledge graph enhanced text generation technology, the influence factor I of the entity e and relationship r in the knowledge graph on text generation er The formula Calculate, where count(e,r) is the number of occurrences of entity e and relation r in the knowledge graph.

[0063] In this invention, the user interaction module adds a sentiment analysis function. It identifies the emotional state by analyzing the language expression and operation behavior of the user's interaction with the system. Using the sentiment classification model, taking text sentiment analysis as an example, the model outputs the probability of category y where z y is the score of category y. The interaction strategy is adjusted according to the sentiment analysis result S. If S is dissatisfied, the probability P of the system automatically providing a solution to the problem solution The formula Calculation, time unsolved Time is the time when the problem is not solved. total is the total interaction time.

[0064] In this invention, the declaration process automation module introduces quantum encryption technology to ensure data transmission security. Quantum key distribution adopts BB84 protocol, and the key generation rate R can be calculated by formula Calculation, p is the probability of single photon emission, η is the detector efficiency. The application process is managed by smart contracts, and the contract execution time is T execute It can be expressed as Where T stepi The execution time of step i is defined in the contract. The rights and obligations of each party are clearly defined in the contract, and the contract is automatically executed when the preset conditions are met, thus improving the fairness and transparency of the declaration.

[0065] The above are only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A multi-modal data fusion enterprise benefit policy declaration automatic processing system, characterized by: Includes the following modules: Multimodal data collection module: adopts a combination of active and passive collection mechanisms to obtain data from within the enterprise, uses web crawlers to capture public information, uses digital watermarks to mark data, and uses a weighted coefficient method to assign weights to data from different sources; Data preprocessing module: proposes data cleaning and normalization strategies, uses a combination of rules and machine learning algorithms to detect outliers for structured data, processes unstructured data through semantic analysis, and uses a dynamic range adjustment formula for feature normalization; Multimodal data fusion module: Design a multimodal neural network architecture, combine CNN, RNN and self-attention mechanism, and realize data fusion through multi-head attention calculation formula; Policy matching module: Builds a policy matching engine based on knowledge graph and deep learning, uses graph embedding and twin networks to calculate similarity, introduces a dynamic update mechanism, and adjusts the matching strategy according to the policy update time; Application materials generation module: This module uses a template-driven and natural language generation (NLG) approach to generate text based on policies and corporate data, uses a perplexity formula to assess text quality, and checks and corrects text according to grammatical rules and policy requirements. Declaration process automation module: Using blockchain technology, triggering the declaration process through smart contracts, connecting with government systems using API gateway technology, and calculating the data transmission success rate through formulas; Intelligent feedback and optimization module: Utilizes reinforcement learning algorithms to optimize the system, uses Q-learning algorithms to update Q-values ​​through feedback signals from application results, analyzes user operation behaviors, and explores user needs; User interaction module: Create an immersive interactive interface, integrate VR and AR technologies, support voice interaction and gesture recognition, calculate the accuracy of voice command recognition through formulas, and dynamically adjust the interface layout and interaction methods based on user habits and preferences.

2. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: Also includes: Policy recommendation optimization module: Use transfer learning technology to transfer model parameters, the new model parameters θ new Denoted as θ new =(1-β)θ old +βθ train ,θ old is the old model parameter, θ train is the parameter for new task training, and β is the migration coefficient. Combining enterprise development strategy and market trend forecast, the Bayesian network model is used to evaluate the potential benefits E of the policy. The formula is: P(S i ) is the policy in state S i The probability of occurrence under i ) is the profit in this state.

3. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: Also includes: Multimodal data quality assessment module: Introducing the fuzzy comprehensive evaluation method, assuming that the data quality assessment index set U = {u1,u2,…,u m }, comment set V = {v1,v2,…,v n }, fuzzy relationship matrix R=(r ij ) m×n , weight vector A=(a1,a2,…,a m ), the comprehensive evaluation result B=A○R, ○ is the fuzzy synthesis operator, and the maximum-minimum synthesis method is used. j = 1, 2, ..., n; the data is graded according to the scores, with a focus on marking and repairing low-quality data.

4. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: In the multimodal data acquisition module, the weighted coefficient method is used to allocate the importance of data from different sources. i Weight w i The calculation formula is f i For data D i Credibility factor; Increase the data collection of enterprise IoT devices, obtain equipment operation status information by integrating with production equipment and sensors, and use federated learning technology. The local model update formula is: is the model parameter of the i-th enterprise in the t-th round, α is the learning rate, L is the loss function, D i is the local data of the i-th enterprise, and the global model parameters w i is the weight of the i-th enterprise, and k is the number of enterprises.

5. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: In the data preprocessing module, structured data is combined with rule-based and machine learning algorithms to detect data anomalies through the local outlier factor LOF formula, which is: p is the data point, N k (p) is the k-neighborhood of p, lrd is the local reachability density; unstructured data uses feature normalization Dynamic range adjustment, is the dynamic mean of the data, is the dynamic standard deviation, m is the number of data samples; Combine semi-supervised learning and active learning to label unstructured data. Semi-supervised learning uses labeled data D label and unlabeled data D unlabel Training model, model loss function L = L label (D label )+λL unlabel (D unlabel ), L label and L unlabel are the loss functions for labeled and unlabeled data respectively, and λ is the balance coefficient; Active learning selects the most valuable data annotations based on model uncertainty. The uncertainty U(x) is expressed by the entropy formula Calculate, P(y i |x) is the sample x belonging to category y i The probability of c is the number of categories; the knowledge distillation technology is introduced to transfer the knowledge of large pre-trained language models to lightweight models, and the distillation loss function z s and z t are the outputs of the student model and the teacher model respectively, T is the temperature parameter, and KL is the KL divergence.

6. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: The multi-head attention formula in the multimodal data fusion module is MultiHead(Q,K,V)=Concat(head1,…,head,)w9, where Q, K, and V are query, key, and value matrices respectively, d k is the dimension of the key vector, and W O is the learning weight matrix; Construct a cross-modal attention network with the attention weight w between different modalities m and n mn The calculation formula is s mn is the similarity score between the features of modality m and n, calculated by dot product f m and f n are the feature vectors of modalities m and n respectively; the adversarial training mechanism is introduced, and the loss functions of the generator G and the discriminator D are L G = -log(D(G(z))) and L D =-log(D(x))-log(1-D(G(z))), x is the real data and z is the random noise.

7. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: The TransE model is used in the policy matching module. The embedding vectors of the relationship r corresponding to the entities h and t satisfy h+r≈t. The similarity sim uses the cosine similarity formula A and B are vector representations of enterprise data and policy conditions, respectively; Using meta-learning technology, the meta-update formula is θ is the meta-model parameter, β is the meta-learning rate, For task T i The loss function is α is the inner loop learning rate, S i For task T i support set.

8. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: The perplexity formula is used to evaluate the quality of text generation in the application materials generation module. W=w1,…,w N is the generated text sequence, P(w i |w1,…,w i-1 ) is the probability of predicting the i-th word based on the language model; The application materials are generated by combining the generative adversarial network (GAN) and the variational autoencoder (VAE). GAN generates diversified texts, while VAE controls the semantic constraints and quality of the generated texts. The loss function of VAE is L VAE =L recon +λL KL , L recon is the reconstruction loss, L KL is the KL divergence loss; the knowledge graph enhanced text generation technology is introduced, and the influence factor I of the entity e and relationship r in the knowledge graph on text generation er By formula Calculate, count(e,r) is the number of times entity e and relation r appear in the knowledge graph.

9. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: The accuracy of voice command recognition in the user interaction module Acc is calculated by the formula Calculation, R correct is the number of correctly recognized instructions, R total is the total number of instructions; Add sentiment analysis function, identify the emotional state by analyzing the language expression and operation behavior of the user's interaction with the system, and use the sentiment classification model. The model outputs the probability of category y z y is the score of category y; adjust the interaction strategy according to the sentiment analysis result S. If S is dissatisfied, the probability P of the system automatically providing a solution to the problem solution By formula Calculation, time unsolved Time is the time when the problem is not solved. total is the total interaction time.

10. The automated processing system for enterprise benefit policy declaration based on multimodal data fusion according to claim 1 is characterized in that: Smart contract execution conditions in the declaration process automation module C i For the i-th condition, the data transmission success rate S is calculated by the formula Calculation, M success is the amount of data successfully transmitted, M total is the total amount of data transmitted; Introducing quantum encryption technology, quantum key distribution uses the BB84 protocol, and the key generation rate R is calculated by the formula Calculation, p is the probability of single photon emission, η is the detector efficiency; the application process is managed by smart contracts, and the contract execution time T stepi is the execution time of the i-th step.

Images