Grassroots social governance intelligent decision support system and method based on multi-modal fusion

Abstract

The invention discloses an intelligent decision support system and method for grassroots social governance based on multi-modal fusion. The method comprises the following steps: S1, acquiring and preprocessing multi-modal data in the field of grassroots social governance; s2, constructing a heterogeneous graph structure according to the data type and the treatment subject category; s3, extracting single-modal features respectively and generating cross-modal dynamic association features; s4, performing collaborative optimization on the heterogeneous graph structure and the cross-modal fusion parameters by adopting a flying fox optimization algorithm; s5, analyzing data in real time according to the optimized heterogeneous graph, and generating potential risk early warning, event trend prediction and event association deduction information; s6, pushing an auxiliary decision-making scheme and a disposal strategy in real time; and S7, continuously optimizing the heterogeneous graph structure by using a feedback result. The decision-making efficiency and accuracy of grassroots social governance are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of social governance integration technology, and in particular to a grassroots social governance intelligent decision support system and method based on multimodal fusion. Background Art

[0002] With the rapid development of information technology and artificial intelligence (AI), grassroots social governance is gradually shifting from traditional empirical decision-making to intelligent decision-making driven by data. Grassroots social governance involves multiple aspects, including social risk management, emergency prediction, and response. The governance data exhibits distinct multimodal characteristics, including both structured and semi-structured data. Existing intelligent decision-making solutions for grassroots social governance typically employ single-modal or simple multimodal data analysis techniques, such as text mining, image recognition, traditional machine learning models, or single-modal deep learning models, aiming to achieve basic risk prediction, event analysis, and governance decision-making support.

[0003] Existing grassroots governance decision-making methods generally provide decision support through simple multi-source data integration and statistical analysis. For example, they use rule-based or predefined model-based methods to achieve data integration and analysis, or use network analysis methods based on traditional graph structures to conduct correlation deduction and risk prediction of governance events. However, these methods generally have problems such as low data fusion, single data feature expression, and limited model generalization capabilities. In particular, they lack in-depth mining and dynamic analysis of complex correlations between multimodal heterogeneous data. In addition, due to the heterogeneity of data and the complex spatiotemporal distribution characteristics of multi-source data, traditional optimization methods find it difficult to accurately and effectively optimize the parameters of the dynamic characteristics of heterogeneous data networks, which in turn affects the real-time and accuracy of decision-making, making it difficult to meet the high requirements of grassroots social governance scenarios for real-time and accurate decision-making responses.

[0004] In recent years, graph neural network technology has gradually gained attention due to its significant advantages in mining complex graph data features. However, existing graph neural network technologies are still mostly based on homogeneous graph networks and lack effective processing methods for multimodal heterogeneous data. At the same time, the Feihu optimization algorithm, as an emerging intelligent optimization method, is gradually being used in various data-driven decision-making scenarios due to its rapid convergence and parameter optimization capabilities. However, in the field of grassroots governance, there is still a lack of in-depth integration of graph neural networks and the Feihu optimization algorithm to fully exploit the inherent correlations and dynamic characteristics of heterogeneous data to provide efficient decision-making support methods.

[0005] Therefore, how to provide an intelligent decision-making support system and method for grassroots social governance based on multimodal fusion is an issue that technical personnel in this field urgently need to solve. Summary of the Invention

[0006] One purpose of the present invention is to propose an intelligent decision-making support method for grassroots social governance based on multimodal fusion. The present invention has the advantages of precise data fusion, real-time decision response, and accurate risk prediction, which effectively improves the efficiency of governance decision-making.

[0007] The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to an embodiment of the present invention is characterized by comprising the following steps:

[0008] S1. Acquire text data, image data, video data, audio data, social media data, and IoT perception data in the field of grassroots social governance, and pre-process them to form structured and semi-structured data sets;

[0009] S2. Divide structured and semi-structured data sets into nodes based on data types and governance subject categories, and construct a heterogeneous graph structure with multi-level dynamic interactive relationships;

[0010] S3. Based on the heterogeneous graph structure, features within a single data modality are extracted separately, and features are interactively fused between different data modalities to generate cross-modal dynamic correlation features;

[0011] S4. Using the Feihu optimization algorithm, with the accuracy and real-time performance of grassroots governance decision-making as the optimization goals, the heterogeneous graph structure and the parameters of cross-modal feature fusion based on cross-modal dynamic correlation features are collaboratively searched and optimized to obtain the optimized heterogeneous graph structure;

[0012] S5. Based on the optimized heterogeneous graph structure, perform comprehensive analysis on structured and semi-structured data sets in real time to generate potential social risk warning information, event trend prediction information, and event correlation relationship deduction information;

[0013] S6. Based on social risk warning information, event trend forecast information, and event correlation relationship deduction information, real-time push of auxiliary decision-making plans and disposal strategies for grassroots social governance;

[0014] S7. Input the feedback results of the actual execution of the auxiliary decision-making plan and the disposal strategy into the Feihu optimization algorithm to dynamically update and continuously optimize the heterogeneous graph structure.

[0015] Optionally, the S2 specifically includes:

[0016] S21. Predefine multiple independent governance theme subgraphs based on governance theme categories, and map structured data and semi-structured data sets into corresponding governance theme subgraphs, with each data node corresponding to a governance theme;

[0017] S22. Determine the spatial distance relationship between the data nodes based on the geographic location coordinates of the data nodes included in the structured data and semi-structured data sets, cluster the nodes using a preset spatial distance threshold, and divide the data nodes whose spatial distance is less than the threshold into a set of spatial clustered nodes;

[0018] S23. For each data node in the spatial cluster node set, determine the degree of overlap of data collection time intervals between the data nodes based on the collection start and end times of the data nodes, and further aggregate the data nodes whose collection time interval overlap exceeds the threshold using a preset time overlap ratio threshold as a standard to form a spatiotemporal cluster node set;

[0019] S24, performing semantic analysis on the data node content in the spatiotemporal clustering node set, extracting keywords from the node content and calculating the co-occurrence of keywords between nodes, and establishing cross-modal semantic association relationships between data nodes of different data types based on the number of co-occurrences of keywords exceeding a preset threshold;

[0020] S25. Perform semantic analysis on the content of data nodes in multiple different governance theme subgraphs, extract and count the number of data nodes with common semantic keywords between different governance themes, and determine the topic association relationship between the nodes of different governance theme subgraphs based on the number of common keyword data nodes exceeding a preset threshold;

[0021] S26. Integrate the spatial clustering node set, spatiotemporal clustering node set, cross-modal semantic association relationship and cross-governance theme theme association relationship, and superimpose them step by step to form a heterogeneous graph structure with multi-dimensional comprehensive interaction of space, time, semantics and themes.

[0022] Optionally, the S3 specifically includes:

[0023] S31. Based on the constructed heterogeneous graph structure, for a single data modality data node in each governance theme subgraph, a set of semantic primitives for representing semantic content features is predefined. By calculating the item-by-item semantic similarity between the content of each data node and the set of semantic primitives, initial semantic features representing the semantic differences of the data nodes are extracted.

[0024] S32, based on the spatial clustering node set, determining a set of data nodes in a spatial neighborhood, and constructing a spatial semantic difference feature reflecting the spatial semantic difference node by node based on the initial semantic feature difference between the data node and other data nodes in the spatial neighborhood;

[0025] S33. Construct a set of adjacent time series data nodes based on the time sequence of data node collection, determine the evolution direction of the node semantic content based on the initial semantic feature change trend between the data node and the adjacent time series data nodes, and generate dynamic time series features reflecting the evolution of the data node over time for each node;

[0026] S34, by performing cross-modal semantic enhancement on the initial semantic features step by step, dynamically expanding the cross-modal fusion range, and generating cross-modal semantic enhancement features;

[0027] S35. For the enhanced cross-modal semantic features, by constructing a cross-modal semantic feature consistency evaluation matrix, the semantic collaboration relationship between each data node and different modal nodes is determined, and the collaboration weight of the cross-modal semantic features is adjusted in real time based on the semantic collaboration relationship, forming a real-time adjustable cross-modal semantic collaboration feature;

[0028] S36. Integrate node spatial semantic difference features, dynamic temporal features, and real-time adjusted cross-modal semantic collaboration features, perform joint feature fusion and aggregation updates node by node, and finally form cross-modal dynamic correlation features.

[0029] Optionally, the S34 specifically includes:

[0030] S341. For a single data modality node set within each governance theme subgraph, dynamically determine multiple semantic expression typical nodes from each data modality node set as semantic feature anchor points based on the stability and representativeness of the node's initial semantic features. The typical nodes are determined by calculating the difference in semantic stability between the node's initial semantic features and the initial semantic features of other nodes within the modality.

[0031] S342, using the initial semantic features of the semantic feature anchor point as a benchmark, calculating the semantic matching degree between each node in the data modality node set and the corresponding modality semantic feature anchor point, and constructing a modality internal semantic matching structure between the nodes in each data modality node set and the semantic feature anchor point;

[0032] S343. Based on the degree of structural similarity between the semantic matching structures within the modalities, determine the data modality pair with the highest degree of structural similarity as the primary fusion starting modality, and adopt a structural similarity quantification standard, which is calculated based on the similarity of the semantic matching degree distribution between the nodes within the modality and the anchor points;

[0033] S344, performing cross-modal semantic feature fusion on the data nodes of the primary fusion starting modality according to the modality internal semantic matching structure, gradually forming primary cross-modal semantic features, and based on the fused features, calculating in real time the cross-modal semantic compatibility index between the unfused data modality nodes and the fused features;

[0034] S345. Determine the data modality node set to be fused in the next stage using the cross-modal semantic compatibility index, introduce the next data modality node set step by step in descending order of the cross-modal semantic compatibility index, update the cross-modal semantic compatibility index in real time each time a data modality node set is introduced, and dynamically perform enhanced cross-modal semantic feature fusion to gradually expand the scope of cross-modal fusion.

[0035] S346. Repeat the cross-modal semantic enhancement fusion step by step until all data modal node sets are fused, and finally generate cross-modal semantic enhancement features.

[0036] Optionally, the S4 specifically includes:

[0037] S41. Taking the accuracy and real-time performance of grassroots governance decision-making as the optimization goals, the node connection relationship of the heterogeneous graph structure, the cross-modal feature fusion weights, and the feature embedding dimensions are encoded separately to form an optimization parameter set;

[0038] S42. Based on historical decision-making data and actual governance results in grassroots governance scenarios, a feedback knowledge base is established to guide the search direction of the Feihu optimization algorithm. The feedback knowledge base dynamically stores and updates historical parameter optimization performance in real time.

[0039] S43. Based on the historical parameter optimization performance in the feedback knowledge base, perform fitness evaluation on the initial heterogeneous graph structure optimization parameter set, and output the fitness value of each optimization parameter set;

[0040] S44, sorting the fitness values ​​of the optimized parameter sets, and selecting the parameter sets with the highest fitness as the initial reference parameter sets for the next step of optimization;

[0041] S45. Using the Feihu optimization algorithm, dynamically update the search position of the optimization parameter set according to the parameter numerical performance of the initial reference parameter set and the historical optimization trajectory, and form a new heterogeneous graph structure optimization parameter set;

[0042] S46. Perform fitness evaluation on the new heterogeneous graph structure optimization parameter set again, and feed back the evaluation results to the feedback knowledge base in real time. Continue to loop fitness evaluation, parameter update, and feedback knowledge base update until the preset optimization termination condition is met. Output the final optimized heterogeneous graph structure parameter set to form an optimized heterogeneous graph structure.

[0043] Optionally, the S5 specifically includes:

[0044] S51. Based on the optimized heterogeneous graph structure, calculate the semantic connection strength change trend between data nodes in real time, identify and determine the core semantic connection path at the current moment, and output the data node set with the highest semantic connection strength change trend as the key node group in real time;

[0045] S52. Analyze the semantic sensitivity of each key node group one by one, identify nodes with abnormal semantic fluctuations in real time, and expand from the abnormally fluctuating nodes to neighboring nodes, determine the semantic conflict diffusion path and the potential social risk impact range, and generate potential social risk warning information in spatial and semantic dimensions in real time;

[0046] S53, obtaining the current evolution state of the semantic features of the key node group and its surrounding nodes in the heterogeneous graph structure in real time, determining the node semantic evolution trajectory at future consecutive moments based on the real-time semantic evolution speed and direction between the nodes, and generating event trend prediction information for each node;

[0047] S54. Based on the current real-time node semantic evolution trajectory, determine the cross-node group semantic evolution path triggered between key node groups in a short time sequence, analyze the triggering conditions of the interaction of the semantic features of the node groups, output the associated trigger nodes, impact paths and evolution sequences between events, and form complete and continuous deduction information on the association relationship between events;

[0048] S55. Establish a real-time monitoring mechanism for node semantic emergencies. By comparing the current node semantic state with the historical state change rate in real time, automatically capture sudden abnormal changes in node semantics, and automatically include sudden abnormal nodes in the early warning node set, and update potential social risk early warning information and event trend prediction information in real time.

[0049] Optionally, the S6 specifically includes:

[0050] S61. Based on potential social risk warning information, event trend forecast information, and event correlation relationship deduction information, a multi-dimensional decision feature space containing risk level, event evolution rate, and probability of triggering related events is constructed in real time, and key decision factors of the current governance situation are dynamically determined;

[0051] S62. Establish a multidimensional similarity evaluation matrix between current key decision-making factors and historical grassroots governance decision-making samples in real time. By calculating the multidimensional similarity between the key decision-making factor characteristics of the current situation and the characteristics of historical governance decision-making samples, automatically select the historical governance decision-making samples that have the highest degree of match with the current governance situation.

[0052] S63. Analyze the governance effectiveness of each disposal measure in the sample after implementation based on the real-time screening of historical governance decision samples. Automatically generate an initial ranking of auxiliary decision measures based on the sample's governance effectiveness and the real-time similarity between the sample and the current governance context.

[0053] S64. Real-time tracking of the dynamic evolution trend of key decision-making factors in the current situation. Based on the real-time change direction and rate of change of key decision-making factors, automatic dynamic adjustment of the initial auxiliary decision-making measures ranking, and real-time generation of dynamic auxiliary decision-making solutions that are consistent with the current governance situation.

[0054] S65. Using the current dynamic decision-making support plan as input, perform real-time online simulation of the effectiveness of governance measures, automatically evaluate the real-time evolution trends of different governance measure combinations on key decision factors, and dynamically determine the optimal governance and disposal strategy sequence;

[0055] S66. Push the optimal governance and disposal strategy sequence confirmed by online simulation to the grassroots governance decision-making terminal in real time, forming auxiliary decision-making plans and governance and disposal strategies that are dynamically updated in real time and adapt to rapid changes in governance situations.

[0056] Optionally, the S64 specifically includes:

[0057] S641. Real-time extraction of multi-dimensional dynamic trend characteristics of key decision-making factors in the current governance context, including risk level change trends, event evolution rate change trends, and associated event trigger probability change trends, and establish trend evolution trajectories for each decision-making factor;

[0058] S642. Analyze the trend evolution trajectory of each key decision factor one by one, determine the trend evolution inflection point of each decision factor and the possible time of the inflection point in real time, and automatically generate the stability and change sensitivity index of the trend state of each decision factor;

[0059] S643. Based on the trend stability and change sensitivity indicators of key decision factors, a real-time matching evaluation model between decision-making support measures and key decision factors is constructed to evaluate in real time the degree of fit between each decision-making support measure and the current changing state of the decision factors.

[0060] S644. Through the real-time matching evaluation model, determine the real-time trend fit score of each decision-making support measure in the current governance context, and dynamically quantify the immediate adaptability of the decision-making support measures;

[0061] S645. Dynamically adjust the priority sequence of the initial decision-making support measures one by one based on the change in trend fit scores in real time, prioritizing the decision-making support measures with the highest trend fit scores at the front of the sequence, and forming a dynamic priority ranking of measures that adapts to the trend changes of the current decision-making factors in real time;

[0062] S646. Monitor the changes in the trend evolution trajectory of key decision-making factors in real time, automatically update the trend evolution trajectory, trend stability and sensitivity indicators, and real-time matching evaluation models, and dynamically generate auxiliary decision-making plans that continuously match the current governance context.

[0063] Optionally, the S7 specifically includes:

[0064] S71. Establish a semantic response model between feedback data of actual grassroots governance disposal strategies and expected governance scenarios in real time. Automatically identify semantic anomalies in governance effects by calculating the multi-dimensional semantic difference vectors between actual feedback results and governance scenarios.

[0065] S72. For identified semantic anomalies, a governance effect difference space is constructed in real time based on the semantic difference vector. The core nodes and key connection paths that cause the governance effect to deviate from the expected target are determined through the real-time distribution characteristics of the semantic difference values ​​of each dimension in the difference space.

[0066] S73. Conduct sensitivity analysis on the semantic difference value change trends of core nodes and key connection paths in real time. By quantifying the sensitivity of nodes and connection paths to governance effects in real time, identify nodes and connection relationships in the heterogeneous graph structure that urgently need to be updated.

[0067] S74, using the core nodes and key connection paths determined by the sensitivity analysis as anchor points for dynamic optimization of the heterogeneous graph structure, calculating in real time the semantic feature offset between other nodes in the heterogeneous graph and the anchor nodes, and automatically determining the connection update area of ​​the heterogeneous graph structure based on the offset;

[0068] S75. Perform a local dynamic reconstruction process of the heterogeneous graph structure connection relationship in real time based on the semantic feature offset, and automatically update the weight parameters of the feature fusion between nodes, so that the heterogeneous graph structure can dynamically adapt to the changes in the governance context reflected by the actual feedback results in real time;

[0069] S76. Feedback the locally dynamically reconstructed heterogeneous graph structure and updated node fusion weight parameters into the Feihu optimization algorithm in real time, automatically triggering the Feihu optimization algorithm to start a new round of overall optimization process of the heterogeneous graph structure, and continuously and dynamically adjust the governance decision-making method.

[0070] Optional, grassroots social governance intelligent decision support system based on multimodal fusion, including the following modules:

[0071] The data collection and preprocessing module is used to collect and preprocess text data, image data, video data, audio data, social media data, and IoT perception data in the field of grassroots social governance to generate structured and semi-structured data sets;

[0072] Heterogeneous graph construction module, used to construct a heterogeneous graph structure with dynamic interactive relationships in multiple dimensions of space, time, semantics and themes based on governance topic categories and data types;

[0073] The cross-modal feature fusion module is used to extract features within a single data modality based on heterogeneous graph structures and perform cross-modal feature interactive fusion to generate cross-modal dynamic correlation features;

[0074] The heterogeneous graph optimization module uses the Feihu optimization algorithm to optimize the accuracy and real-time performance of grassroots governance decisions. It performs collaborative search and optimization on heterogeneous graph structures and cross-modal feature fusion parameters to obtain the optimized heterogeneous graph structure.

[0075] The risk prediction and deduction module is used to generate potential social risk warning information, event trend prediction information, and event correlation relationship deduction information in real time based on the optimized heterogeneous graph structure;

[0076] A decision-making support module is used to dynamically generate decision-making support plans and governance strategies that are highly compatible with the current governance context in real time based on potential social risk warning information, event trend forecast information, and information on the correlation between events.

[0077] The feedback self-evolution module is used to input the feedback results of the actual implementation of auxiliary decision-making plans and disposal strategies into the Feihu optimization algorithm in real time, realize the dynamic update and continuous optimization of heterogeneous graph structures, and enable the grassroots social governance intelligent decision-making support system to continue to self-evolve.

[0078] The beneficial effects of the present invention are:

[0079] (1) The present invention realizes deep semantic interaction between different data modalities by constructing a multimodal heterogeneous graph structure and integrating cross-modal dynamic association features, effectively improving the fusion accuracy of multi-source data and improving the efficiency of data utilization and decision-making accuracy in grassroots social governance decision-making.

[0080] (2) The present invention adopts the Feihu optimization algorithm to perform real-time collaborative optimization of heterogeneous graph structures and cross-modal fusion parameters, which can achieve efficient dynamic optimization of heterogeneous data network parameters, significantly improve the real-time performance of decision-making responses, and show better adaptability in the actual dynamic changing environment of grassroots social governance scenarios.

[0081] (3) In terms of potential risk warning, event trend prediction and event correlation deduction, the present invention effectively solves the problems of single data analysis and insufficient trend prediction accuracy in existing technologies through dynamic trend analysis of key decision-making factors and adaptive auxiliary decision-making solution generation mechanism. It breaks through the response bottleneck of existing grassroots governance technology in rapid situational changes, realizes the self-evolution of governance decision-making methods, and effectively improves the decision-making response capability and disposal effect of grassroots social governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0083] Figure 1 This is a schematic diagram of the overall process of the intelligent decision-making support method for grassroots social governance based on multimodal fusion proposed in the present invention;

[0084] Figure 2 This is a schematic diagram of the heterogeneous graph structure of the grassroots social governance intelligent decision support method based on multimodal fusion proposed in the present invention;

[0085] Figure 3 This is a schematic diagram of the system module structure of the grassroots social governance intelligent decision support system based on multimodal fusion proposed in the present invention. DETAILED DESCRIPTION

[0086] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0087] refer to Figure 1-Figure 3 , an intelligent decision-making support method for grassroots social governance based on multimodal fusion, including the following steps:

[0088] S1. Acquire text data, image data, video data, audio data, social media data, and IoT perception data in the field of grassroots social governance, and pre-process them to form structured and semi-structured data sets;

[0089] S2. Divide structured and semi-structured data sets into nodes based on data types and governance subject categories, and construct a heterogeneous graph structure with multi-level dynamic interactive relationships;

[0090] S3. Based on the heterogeneous graph structure, features within a single data modality are extracted separately, and features are interactively fused between different data modalities to generate cross-modal dynamic correlation features;

[0091] S4. Using the Feihu optimization algorithm, with the accuracy and real-time performance of grassroots governance decision-making as the optimization goals, the heterogeneous graph structure and the parameters of cross-modal feature fusion based on cross-modal dynamic correlation features are collaboratively searched and optimized to obtain the optimized heterogeneous graph structure;

[0092] S5. Based on the optimized heterogeneous graph structure, perform comprehensive analysis on structured and semi-structured data sets in real time to generate potential social risk warning information, event trend prediction information, and event correlation relationship deduction information;

[0093] S6. Based on social risk warning information, event trend forecast information, and event correlation relationship deduction information, real-time push of auxiliary decision-making plans and disposal strategies for grassroots social governance;

[0094] S7. Input the feedback results of the actual execution of the auxiliary decision-making plan and the disposal strategy into the Feihu optimization algorithm to dynamically update and continuously optimize the heterogeneous graph structure.

[0095] By constructing a heterogeneous graph structure with multi-level dynamic interactive relationships, integrating the characteristics of different modal data, and using the Feihu optimization algorithm to collaboratively optimize parameters, we have achieved precise integration and comprehensive analysis of multimodal data in the field of grassroots social governance, effectively improving the accuracy and real-time performance of social risk warning, event trend prediction, and event correlation deduction, thereby enhancing the reliability and adaptability of grassroots social governance decision-making support solutions, and has good practical application value.

[0096] In this embodiment, S2 specifically includes:

[0097] S21. Predefine multiple independent governance theme subgraphs based on governance theme categories, and map structured data and semi-structured data sets into corresponding governance theme subgraphs, with each data node corresponding to a governance theme;

[0098] S22. Determine the spatial distance relationship between the data nodes based on the geographic location coordinates of the data nodes included in the structured data and semi-structured data sets, cluster the nodes using a preset spatial distance threshold, and divide the data nodes whose spatial distance is less than the threshold into a set of spatial clustered nodes;

[0099] S23. For each data node in the spatial cluster node set, determine the degree of overlap of data collection time intervals between the data nodes based on the collection start and end times of the data nodes, and further aggregate the data nodes whose collection time interval overlap exceeds the threshold using a preset time overlap ratio threshold as a standard to form a spatiotemporal cluster node set;

[0100] S24, performing semantic analysis on the data node content in the spatiotemporal clustering node set, extracting keywords from the node content and calculating the co-occurrence of keywords between nodes, and establishing cross-modal semantic association relationships between data nodes of different data types based on the number of co-occurrences of keywords exceeding a preset threshold;

[0101] S25. Perform semantic analysis on the content of data nodes in multiple different governance theme subgraphs, extract and count the number of data nodes with common semantic keywords between different governance themes, and determine the topic association relationship between the nodes of different governance theme subgraphs based on the number of common keyword data nodes exceeding a preset threshold;

[0102] S26. Integrate the spatial clustering node set, spatiotemporal clustering node set, cross-modal semantic association relationship and cross-governance theme theme association relationship, and superimpose them step by step to form a heterogeneous graph structure with multi-dimensional comprehensive interaction of space, time, semantics and themes.

[0103] By constructing a multi-level, multi-dimensional heterogeneous graph structure, and utilizing spatial clustering, spatiotemporal clustering, cross-modal semantic associations, and cross-governance theme associations, the multi-dimensional information of space, time, semantics, and governance themes is effectively integrated, enabling the precise structuring and in-depth correlation analysis of heterogeneous data in the field of grassroots social governance, thereby improving the accuracy and real-time performance of data fusion, and helping to improve the pertinence and effectiveness of grassroots governance decision-making plans.

[0104] In this embodiment, S3 specifically includes:

[0105] S31. Based on the constructed heterogeneous graph structure, for a single data modality data node in each governance theme subgraph, a set of semantic primitives for representing semantic content features is predefined. By calculating the item-by-item semantic similarity between the content of each data node and the set of semantic primitives, initial semantic features representing the semantic differences of the data nodes are extracted.

[0106] S32, based on the spatial clustering node set, determining a set of data nodes in a spatial neighborhood, and constructing a spatial semantic difference feature reflecting the spatial semantic difference node by node based on the initial semantic feature difference between the data node and other data nodes in the spatial neighborhood;

[0107] S33. Construct a set of adjacent time series data nodes based on the time sequence of data node collection, determine the evolution direction of the node semantic content based on the initial semantic feature change trend between the data node and the adjacent time series data nodes, and generate dynamic time series features reflecting the evolution of the data node over time for each node;

[0108] S34, by performing cross-modal semantic enhancement on the initial semantic features step by step, dynamically expanding the cross-modal fusion range, and generating cross-modal semantic enhancement features;

[0109] S35. For the enhanced cross-modal semantic features, by constructing a cross-modal semantic feature consistency evaluation matrix, the semantic collaboration relationship between each data node and different modal nodes is determined, and the collaboration weight of the cross-modal semantic features is adjusted in real time based on the semantic collaboration relationship, forming a real-time adjustable cross-modal semantic collaboration feature;

[0110] S36. Integrate node spatial semantic difference features, dynamic temporal features, and real-time adjusted cross-modal semantic collaboration features, perform joint feature fusion and aggregation updates node by node, and finally form cross-modal dynamic correlation features.

[0111] By comprehensively processing the spatial differences, temporal dynamic evolution, and cross-modal step-by-step enhanced fusion of the semantic features of data nodes in heterogeneous graphs, we have achieved precise fusion and collaborative optimization of multimodal data features in space, time, and cross-modal semantic collaboration, effectively improving the sophistication and accuracy of data analysis, and enhancing the real-time and dynamic adaptability of grassroots social governance decision-making.

[0112] In this embodiment, the S34 specifically includes:

[0113] S341. For a single data modality node set within each governance theme subgraph, dynamically determine multiple semantic expression typical nodes from each data modality node set as semantic feature anchor points based on the stability and representativeness of the node's initial semantic features. The typical nodes are determined by calculating the difference in semantic stability between the node's initial semantic features and the initial semantic features of other nodes within the modality.

[0114] S342, using the initial semantic features of the semantic feature anchor point as a benchmark, calculating the semantic matching degree between each node in the data modality node set and the corresponding modality semantic feature anchor point, and constructing a modality internal semantic matching structure between the nodes in each data modality node set and the semantic feature anchor point;

[0115] S343. Based on the degree of structural similarity between the semantic matching structures within the modalities, determine the data modality pair with the highest degree of structural similarity as the primary fusion starting modality, and adopt a structural similarity quantification standard, which is calculated based on the similarity of the semantic matching degree distribution between the nodes within the modality and the anchor points;

[0116] S344, performing cross-modal semantic feature fusion on the data nodes of the primary fusion starting modality according to the modality internal semantic matching structure, gradually forming primary cross-modal semantic features, and based on the fused features, calculating in real time the cross-modal semantic compatibility index between the unfused data modality nodes and the fused features;

[0117] S345. Determine the data modality node set to be fused in the next stage using the cross-modal semantic compatibility index, introduce the next data modality node set step by step in descending order of the cross-modal semantic compatibility index, update the cross-modal semantic compatibility index in real time each time a data modality node set is introduced, and dynamically perform enhanced cross-modal semantic feature fusion to gradually expand the scope of cross-modal fusion.

[0118] S346. Repeat the cross-modal semantic enhancement fusion step by step until all data modal node sets are fused, and finally generate cross-modal semantic enhancement features.

[0119] By constructing a modal internal semantic matching structure and adopting a cross-modal semantic enhancement mechanism with step-by-step fusion, the cross-modal fusion scope is dynamically expanded, and the efficient fusion and gradual convergence of semantic features between nodes of different data modalities are achieved, which effectively improves the accuracy and robustness of cross-modal data fusion, and further enhances the accuracy and comprehensiveness of grassroots social governance decision-making analysis.

[0120] In this embodiment, the S4 specifically includes:

[0121] S41. Taking the accuracy and real-time performance of grassroots governance decision-making as the optimization goals, the node connection relationship of the heterogeneous graph structure, the cross-modal feature fusion weights, and the feature embedding dimensions are encoded separately to form an optimization parameter set;

[0122] S42. Based on historical decision-making data and actual governance results in grassroots governance scenarios, a feedback knowledge base is established to guide the search direction of the Feihu optimization algorithm. The feedback knowledge base dynamically stores and updates historical parameter optimization performance in real time.

[0123] S43. Based on the historical parameter optimization performance in the feedback knowledge base, perform fitness evaluation on the initial heterogeneous graph structure optimization parameter set, and output the fitness value of each optimization parameter set;

[0124] S44, sorting the fitness values ​​of the optimized parameter sets, and selecting the parameter sets with the highest fitness as the initial reference parameter sets for the next step of optimization;

[0125] S45. Using the Feihu optimization algorithm, dynamically update the search position of the optimization parameter set according to the parameter numerical performance of the initial reference parameter set and the historical optimization trajectory, and form a new heterogeneous graph structure optimization parameter set;

[0126] S46. Perform fitness evaluation on the new heterogeneous graph structure optimization parameter set again, and feed back the evaluation results to the feedback knowledge base in real time. Continue to loop fitness evaluation, parameter update, and feedback knowledge base update until the preset optimization termination condition is met. Output the final optimized heterogeneous graph structure parameter set to form an optimized heterogeneous graph structure.

[0127] By establishing a real-time dynamically updated feedback knowledge base and fitness evaluation mechanism, and combining the Feihu optimization algorithm to dynamically and continuously coordinate the optimization parameters of heterogeneous graph structures, the adaptive ability and continuous optimization level of the grassroots governance decision-making system for feedback on actual governance effects are effectively improved, and the long-term self-evolution of heterogeneous graph structures and grassroots governance decision-making methods is achieved, thereby improving the stability and adaptability of the decision-making system.

[0128] In this embodiment, the S5 specifically includes:

[0129] S51. Based on the optimized heterogeneous graph structure, calculate the semantic connection strength change trend between data nodes in real time, identify and determine the core semantic connection path at the current moment, and output the data node set with the highest semantic connection strength change trend as the key node group in real time;

[0130] S52. Analyze the semantic sensitivity of each key node group one by one, identify nodes with abnormal semantic fluctuations in real time, and expand from the abnormally fluctuating nodes to neighboring nodes, determine the semantic conflict diffusion path and the potential social risk impact range, and generate potential social risk warning information in spatial and semantic dimensions in real time;

[0131] S53, obtaining the current evolution state of the semantic features of the key node group and its surrounding nodes in the heterogeneous graph structure in real time, determining the node semantic evolution trajectory at future consecutive moments based on the real-time semantic evolution speed and direction between the nodes, and generating event trend prediction information for each node;

[0132] S54. Based on the current real-time node semantic evolution trajectory, determine the cross-node group semantic evolution path triggered between key node groups in a short time sequence, analyze the triggering conditions of the interaction of the semantic features of the node groups, output the associated trigger nodes, impact paths and evolution sequences between events, and form complete and continuous deduction information on the association relationship between events;

[0133] S55. Establish a real-time monitoring mechanism for node semantic emergencies. By comparing the current node semantic state with the historical state change rate in real time, automatically capture sudden abnormal changes in node semantics, and automatically include sudden abnormal nodes in the early warning node set, and update potential social risk early warning information and event trend prediction information in real time.

[0134] By analyzing the changing trends of the semantic connection strength of nodes in heterogeneous graph structures in real time, accurately identifying key node groups and conducting sensitivity analysis and trend prediction, we can achieve refined early warning of potential social risks and accurate prediction of event evolution trends, effectively improving the accuracy of risk identification and the timeliness of event deduction, and further enhancing the accuracy and proactive response capabilities of grassroots social governance decision-making.

[0135] In this embodiment, S6 specifically includes:

[0136] S61. Based on potential social risk warning information, event trend forecast information, and event correlation relationship deduction information, a multi-dimensional decision feature space containing risk level, event evolution rate, and probability of triggering related events is constructed in real time, and key decision factors of the current governance situation are dynamically determined;

[0137] S62. Establish a multidimensional similarity evaluation matrix between current key decision-making factors and historical grassroots governance decision-making samples in real time. By calculating the multidimensional similarity between the key decision-making factor characteristics of the current situation and the characteristics of historical governance decision-making samples, automatically select the historical governance decision-making samples that have the highest degree of match with the current governance situation.

[0138] S63. Analyze the governance effectiveness of each disposal measure in the sample after implementation based on the real-time screening of historical governance decision samples. Automatically generate an initial ranking of auxiliary decision measures based on the sample's governance effectiveness and the real-time similarity between the sample and the current governance context.

[0139] S64. Real-time tracking of the dynamic evolution trend of key decision-making factors in the current situation. Based on the real-time change direction and rate of change of key decision-making factors, automatic dynamic adjustment of the initial auxiliary decision-making measures ranking, and real-time generation of dynamic auxiliary decision-making solutions that are consistent with the current governance situation.

[0140] S65. Using the current dynamic decision-making support plan as input, perform real-time online simulation of the effectiveness of governance measures, automatically evaluate the real-time evolution trends of different governance measure combinations on key decision factors, and dynamically determine the optimal governance and disposal strategy sequence;

[0141] S66. Push the optimal governance and disposal strategy sequence confirmed by online simulation to the grassroots governance decision-making terminal in real time, forming auxiliary decision-making plans and governance and disposal strategies that are dynamically updated in real time and adapt to rapid changes in governance situations.

[0142] By constructing a multi-dimensional decision feature space and combining the multi-dimensional similarity evaluation and online simulation analysis of historical decision samples in real time, dynamic decision-making support plans and optimized governance disposal strategy sequences are automatically generated, achieving precise matching and real-time dynamic adjustment between decision-making support measures and actual governance situations, effectively improving the applicability and response efficiency of governance decision-making plans, and significantly enhancing the accuracy and adaptability of grassroots social governance processes.

[0143] In this embodiment, the S64 specifically includes:

[0144] S641. Real-time extraction of multi-dimensional dynamic trend characteristics of key decision-making factors in the current governance context, including risk level change trends, event evolution rate change trends, and associated event trigger probability change trends, and establish trend evolution trajectories for each decision-making factor;

[0145] S642. Analyze the trend evolution trajectory of each key decision factor one by one, determine the trend evolution inflection point of each decision factor and the possible time of the inflection point in real time, and automatically generate the stability and change sensitivity index of the trend state of each decision factor;

[0146] S643. Based on the trend stability and change sensitivity indicators of key decision factors, a real-time matching evaluation model between decision-making support measures and key decision factors is constructed to evaluate in real time the degree of fit between each decision-making support measure and the current changing state of the decision factors.

[0147] S644. Through the real-time matching evaluation model, determine the real-time trend fit score of each decision-making support measure in the current governance context, and dynamically quantify the immediate adaptability of the decision-making support measures;

[0148] S645. Dynamically adjust the priority sequence of the initial decision-making support measures one by one based on the change in trend fit scores in real time, prioritizing the decision-making support measures with the highest trend fit scores at the front of the sequence, and forming a dynamic priority ranking of measures that adapts to the trend changes of the current decision-making factors in real time;

[0149] S646. Monitor the changes in the trend evolution trajectory of key decision-making factors in real time, automatically update the trend evolution trajectory, trend stability and sensitivity indicators, and real-time matching evaluation models, and dynamically generate auxiliary decision-making plans that continuously match the current governance context.

[0150] By establishing the multi-dimensional dynamic trend characteristics and trend evolution trajectories of decision-making factors in real time, constructing trend stability and sensitivity indicators and real-time matching evaluation models, and dynamically adjusting the priority ranking of auxiliary decision-making measures, efficient real-time adaptation of decision-making measures and the evolution of governance situations is achieved, effectively improving the real-time response and dynamic adaptability of grassroots social governance decision-making plans in complex situations.

[0151] In this embodiment, the S7 specifically includes:

[0152] S71. Establish a semantic response model between feedback data of actual grassroots governance disposal strategies and expected governance scenarios in real time. Automatically identify semantic anomalies in governance effects by calculating the multi-dimensional semantic difference vectors between actual feedback results and governance scenarios.

[0153] S72. For identified semantic anomalies, a governance effect difference space is constructed in real time based on the semantic difference vector. The core nodes and key connection paths that cause the governance effect to deviate from the expected target are determined through the real-time distribution characteristics of the semantic difference values ​​of each dimension in the difference space.

[0154] S73. Conduct sensitivity analysis on the semantic difference value change trends of core nodes and key connection paths in real time. By quantifying the sensitivity of nodes and connection paths to governance effects in real time, identify nodes and connection relationships in the heterogeneous graph structure that urgently need to be updated.

[0155] S74, using the core nodes and key connection paths determined by the sensitivity analysis as anchor points for dynamic optimization of the heterogeneous graph structure, calculating in real time the semantic feature offset between other nodes in the heterogeneous graph and the anchor nodes, and automatically determining the connection update area of ​​the heterogeneous graph structure based on the offset;

[0156] S75. Perform a local dynamic reconstruction process of the heterogeneous graph structure connection relationship in real time based on the semantic feature offset, and automatically update the weight parameters of the feature fusion between nodes, so that the heterogeneous graph structure can dynamically adapt to the changes in the governance context reflected by the actual feedback results in real time;

[0157] S76. Feedback the locally dynamically reconstructed heterogeneous graph structure and updated node fusion weight parameters into the Feihu optimization algorithm in real time, automatically triggering the Feihu optimization algorithm to start a new round of overall optimization process of the heterogeneous graph structure, and continuously and dynamically adjust the governance decision-making method.

[0158] By establishing a semantic response model between governance effect feedback data and governance context, the semantic differences of governance effects can be identified in real time, and the core nodes and key connection paths that affect the governance effects can be determined. Local reconstruction and parameter optimization of heterogeneous graph structures can be dynamically performed, effectively realizing the adaptive dynamic update of governance decision-making methods, and improving the rapid response and continuous optimization capabilities of the grassroots social governance intelligent decision-making system to changes in actual application scenarios.

[0159] In this implementation, the grassroots social governance intelligent decision support system based on multimodal fusion includes the following modules:

[0160] The data collection and preprocessing module is used to collect and preprocess text data, image data, video data, audio data, social media data, and IoT perception data in the field of grassroots social governance to generate structured and semi-structured data sets;

[0161] Heterogeneous graph construction module, used to construct a heterogeneous graph structure with dynamic interactive relationships in multiple dimensions of space, time, semantics and themes based on governance topic categories and data types;

[0162] The cross-modal feature fusion module is used to extract features within a single data modality based on heterogeneous graph structures and perform cross-modal feature interactive fusion to generate cross-modal dynamic correlation features;

[0163] The heterogeneous graph optimization module uses the Feihu optimization algorithm to optimize the accuracy and real-time performance of grassroots governance decisions. It performs collaborative search and optimization on heterogeneous graph structures and cross-modal feature fusion parameters to obtain the optimized heterogeneous graph structure.

[0164] The risk prediction and deduction module is used to generate potential social risk warning information, event trend prediction information, and event correlation relationship deduction information in real time based on the optimized heterogeneous graph structure;

[0165] A decision-making support module is used to dynamically generate decision-making support plans and governance strategies that are highly compatible with the current governance context in real time based on potential social risk warning information, event trend forecast information, and information on the correlation between events.

[0166] The feedback self-evolution module is used to input the feedback results of the actual implementation of auxiliary decision-making plans and disposal strategies into the Feihu optimization algorithm in real time, realize the dynamic update and continuous optimization of heterogeneous graph structures, and enable the grassroots social governance intelligent decision-making support system to continue to self-evolve.

[0167] By setting up a data collection and preprocessing module, a heterogeneous graph construction module, a cross-modal feature fusion module, a heterogeneous graph optimization module, a risk prediction and deduction module, an auxiliary decision generation module and a feedback self-evolution module, the multimodal precise fusion and dynamic optimization analysis of grassroots social governance data are achieved, and accurate auxiliary decision-making plans and governance strategies are automatically generated, which effectively improves the real-time, accuracy and continuous self-evolution capabilities of the governance system and improves the comprehensive decision-making effect of grassroots social governance.

[0168] Example 1:

[0169] In order to verify the feasibility of the present invention in implementation, the present invention is applied to a grassroots social governance intelligent decision-making platform, and actual verification is carried out on the social risk warning, event trend prediction and event correlation deduction tasks that are daily handled by the grassroots governance departments in a certain area. The daily governance of this area involves multiple data sources such as text, images, videos, audio, social media and Internet of Things perception. Traditional governance decision support methods generally have problems such as low data integration, insufficient real-time analysis results, and low risk prediction accuracy. For example, traditional methods usually use manual rules, single modality data analysis or simple statistical methods to analyze governance data. This method is costly and time-consuming, and has a high false alarm rate and slow response in the risk prediction of complex events, making it difficult to meet the rapid response requirements of grassroots social governance.

[0170] During the actual operation, we first collect text policy documents, event records, community announcements, resident feedback, image surveillance data, on-site photos, video surveillance clips, law enforcement recorder clips, audio records, on-site interview recordings, as well as social media data from community forums and Weibo and IoT data from environmental monitoring in the governance area over the past three months. We then perform denoising, format unification and semantic labeling pre-processing on the collected data to form structured and semi-structured data sets.

[0171] The system divides the data set according to governance themes and constructs governance theme subgraphs; it then performs spatial and spatiotemporal clustering based on the geographic coordinates and collection time of the data to form spatial clustering node sets and spatiotemporal clustering node sets, and further determines the semantic relevance of data nodes through semantic analysis, thereby constructing a heterogeneous graph structure with multi-dimensional fusion of space, time, semantics, and governance themes.

[0172] Subsequently, the heterogeneous graph neural network in the present invention is used to extract the semantic difference features, spatial semantic difference features, and temporal dynamic features of the data nodes, and generate cross-modal dynamic correlation features through cross-modal step-by-step fusion. Then, the Feihu optimization algorithm is used to determine the optimal configuration of the heterogeneous graph structure and fusion parameters with the accuracy and real-time performance of grassroots governance decision-making as the goal. In the actual optimization process, after 25 rounds of optimization iterations, the parameter set of the system heterogeneous graph structure reached convergence, the node embedding dimension was determined to be 64, and the cross-modal fusion weight matrix converged stably.

[0173] The optimized model analyzes new data over the next month in real time, outputting warnings of potential social risks, event trend forecasts, and event correlations. It also uses online simulations to assess the real-time evolution of different governance measures, automatically generating accurate decision-making support solutions and governance strategies. The optimization algorithm is then fed back with feedback from the governance departments' actual decision-making outcomes, further optimizing the heterogeneous graph structure and achieving adaptive dynamic optimization of the model.

[0174] To specifically demonstrate the improvement effect brought about by the present invention, the following table shows the comparative data of event prediction accuracy between the traditional governance decision-making method and the method of the present invention:

[0175] Table 1 Comparison of event prediction accuracy of grassroots governance decision support methods

[0176] Event Type Actual number of occurrences Traditional method prediction times The method of the present invention predicts the number of Community safety incidents 12 7 11 Environmental pollution incidents 9 5 8 Conflicts and disputes 15 9 14 Public utility failure 8 4 7

[0177] It can be clearly seen from the data analysis in Table 1 that the method of the present invention is significantly superior to the traditional method in terms of event prediction accuracy. Taking community safety incidents as an example, there were 12 actual occurrences, and the method of the present invention accurately predicted 11 times, while the traditional method only predicted 7 times, with an accuracy rate increase of more than 33%. In terms of environmental pollution incidents, the prediction accuracy of the method of the present invention increased by 60%. In addition, in the prediction of conflict and dispute incidents, the method of the present invention accurately predicted 14 times, with an accuracy rate increase of more than 55%. These data fully reflect the significant technical advantages of the method of the present invention in data analysis and decision-making support in the field of grassroots social governance, especially in terms of risk prediction accuracy and real-time response.

[0178] A further comparison between traditional methods and the present invention demonstrates significant real-time responsiveness. After a traditional governance decision plan is generated, it must undergo manual review and evaluation before it can be implemented, a process that typically takes three to five working days. However, the present method automatically generates real-time auxiliary decision plans and simulates and evaluates them, generating highly effective and applicable governance measures in just minutes. This significantly shortens decision-making response time and effectively improves grassroots governance departments' ability to rapidly handle emergencies and complex incidents.

[0179] At the same time, the heterogeneous graph structure of the present invention is dynamically updated and continuously optimized after actual feedback results are input. In practice, when a governance measure fails to achieve the expected results after implementation, the feedback results immediately trigger the optimization algorithm, quickly locate the problem node, and perform local optimization and parameter adjustments on the heterogeneous graph. During 30 consecutive days of actual application, the system underwent six dynamic adjustments, achieving significant and gradual improvements in both model prediction accuracy and decision-making adaptability. In particular, the model's prediction error rate decreased by 25% and 20% in the handling of community safety and conflict and dispute incidents, respectively.

[0180] Based on the actual application effects of the above embodiments, the intelligent decision-making support method for grassroots social governance provided by the present invention shows significant technical advantages and practical application value in data fusion, event prediction accuracy, real-time response capability and adaptive optimization, and significantly improves the scientificity and efficiency of grassroots social governance decision-making.

[0181] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. An intelligent decision-making support method for grassroots social governance based on multimodal fusion, characterized by: The steps include: S1. Acquire text data, image data, video data, audio data, social media data, and IoT perception data in the field of grassroots social governance, and pre-process them to form structured and semi-structured data sets; S2. Divide structured and semi-structured data sets into nodes based on data types and governance subject categories, and construct a heterogeneous graph structure with multi-level dynamic interactive relationships; S3. Based on the heterogeneous graph structure, features within a single data modality are extracted separately, and features are interactively fused between different data modalities to generate cross-modal dynamic correlation features; S4. Using the Feihu optimization algorithm, with the accuracy and real-time performance of grassroots governance decision-making as the optimization goals, the heterogeneous graph structure and the parameters of cross-modal feature fusion based on cross-modal dynamic correlation features are collaboratively searched and optimized to obtain the optimized heterogeneous graph structure; S5. Based on the optimized heterogeneous graph structure, perform comprehensive analysis on structured and semi-structured data sets in real time to generate potential social risk warning information, event trend prediction information, and event correlation relationship deduction information; S6. Based on social risk warning information, event trend forecast information, and event correlation relationship deduction information, real-time push of auxiliary decision-making plans and disposal strategies for grassroots social governance; S7. Input the feedback results of the actual execution of the auxiliary decision-making plan and the disposal strategy into the Feihu optimization algorithm to dynamically update and continuously optimize the heterogeneous graph structure.

2. The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to claim 1 is characterized in that: The S2 specifically includes: S21. Predefine multiple independent governance theme subgraphs based on governance theme categories, and map structured data and semi-structured data sets into corresponding governance theme subgraphs, with each data node corresponding to a governance theme; S22. Determine the spatial distance relationship between the data nodes based on the geographic location coordinates of the data nodes included in the structured data and semi-structured data sets, cluster the nodes using a preset spatial distance threshold, and divide the data nodes whose spatial distance is less than the threshold into a set of spatial clustered nodes; S23. For each data node in the spatial cluster node set, determine the degree of overlap of data collection time intervals between the data nodes based on the collection start and end times of the data nodes, and further aggregate the data nodes whose collection time interval overlap exceeds the threshold using a preset time overlap ratio threshold as a standard to form a spatiotemporal cluster node set; S24, performing semantic analysis on the data node content in the spatiotemporal clustering node set, extracting keywords from the node content and calculating the co-occurrence of keywords between nodes, and establishing cross-modal semantic association relationships between data nodes of different data types based on the number of co-occurrences of keywords exceeding a preset threshold; S25. Perform semantic analysis on the content of data nodes in multiple different governance theme subgraphs, extract and count the number of data nodes with common semantic keywords between different governance themes, and determine the topic association relationship between the nodes of different governance theme subgraphs based on the number of common keyword data nodes exceeding a preset threshold; S26. Integrate the spatial clustering node set, spatiotemporal clustering node set, cross-modal semantic association relationship and cross-governance theme theme association relationship, and superimpose them step by step to form a heterogeneous graph structure with multi-dimensional comprehensive interaction of space, time, semantics and themes.

3. The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to claim 1 is characterized in that: The S3 specifically includes: S31. Based on the constructed heterogeneous graph structure, for a single data modality data node in each governance theme subgraph, a set of semantic primitives for representing semantic content features is predefined. By calculating the item-by-item semantic similarity between the content of each data node and the set of semantic primitives, initial semantic features representing the semantic differences of the data nodes are extracted. S32, based on the spatial clustering node set, determining a set of data nodes in a spatial neighborhood, and constructing a spatial semantic difference feature reflecting the spatial semantic difference node by node based on the initial semantic feature difference between the data node and other data nodes in the spatial neighborhood; S33. Construct a set of adjacent time series data nodes based on the time sequence of data node collection, determine the evolution direction of the node semantic content based on the initial semantic feature change trend between the data node and the adjacent time series data nodes, and generate dynamic time series features reflecting the evolution of the data node over time for each node; S34, by performing cross-modal semantic enhancement on the initial semantic features step by step, dynamically expanding the cross-modal fusion range, and generating cross-modal semantic enhancement features; S35. For the enhanced cross-modal semantic features, by constructing a cross-modal semantic feature consistency evaluation matrix, the semantic collaboration relationship between each data node and different modal nodes is determined, and the collaboration weight of the cross-modal semantic features is adjusted in real time based on the semantic collaboration relationship, forming a real-time adjustable cross-modal semantic collaboration feature; S36. Integrate node spatial semantic difference features, dynamic temporal features, and real-time adjusted cross-modal semantic collaboration features, perform joint feature fusion and aggregation updates node by node, and finally form cross-modal dynamic correlation features.

4. The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to claim 3 is characterized in that: The S34 specifically includes: S341. For a single data modality node set within each governance theme subgraph, dynamically determine multiple semantic expression typical nodes from each data modality node set as semantic feature anchor points based on the stability and representativeness of the node's initial semantic features. The typical nodes are determined by calculating the difference in semantic stability between the node's initial semantic features and the initial semantic features of other nodes within the modality. S342, using the initial semantic features of the semantic feature anchor point as a benchmark, calculating the semantic matching degree between each node in the data modality node set and the corresponding modality semantic feature anchor point, and constructing a modality internal semantic matching structure between the nodes in each data modality node set and the semantic feature anchor point; S343. Based on the degree of structural similarity between the semantic matching structures within the modalities, determine the data modality pair with the highest degree of structural similarity as the primary fusion starting modality, and adopt a structural similarity quantification standard, which is calculated based on the similarity of the semantic matching degree distribution between the nodes within the modality and the anchor points; S344, performing cross-modal semantic feature fusion on the data nodes of the primary fusion starting modality according to the modality internal semantic matching structure, gradually forming primary cross-modal semantic features, and based on the fused features, calculating in real time the cross-modal semantic compatibility index between the unfused data modality nodes and the fused features; S345. Determine the data modality node set to be fused in the next stage using the cross-modal semantic compatibility index, introduce the next data modality node set step by step in descending order of the cross-modal semantic compatibility index, update the cross-modal semantic compatibility index in real time each time a data modality node set is introduced, and dynamically perform enhanced cross-modal semantic feature fusion to gradually expand the scope of cross-modal fusion. S346. Repeat the cross-modal semantic enhancement fusion step by step until all data modal node sets are fused, and finally generate cross-modal semantic enhancement features.

5. The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to claim 1 is characterized in that: The S4 specifically includes: S41. Taking the accuracy and real-time performance of grassroots governance decision-making as the optimization goals, the node connection relationship of the heterogeneous graph structure, the cross-modal feature fusion weights, and the feature embedding dimensions are encoded separately to form an optimization parameter set; S42. Based on historical decision-making data and actual governance results in grassroots governance scenarios, a feedback knowledge base is established to guide the search direction of the Feihu optimization algorithm. The feedback knowledge base dynamically stores and updates historical parameter optimization performance in real time. S43. Based on the historical parameter optimization performance in the feedback knowledge base, perform fitness evaluation on the initial heterogeneous graph structure optimization parameter set, and output the fitness value of each optimization parameter set; S44, sorting the fitness values ​​of the optimized parameter sets, and selecting the parameter sets with the highest fitness as the initial reference parameter sets for the next step of optimization; S45. Using the Feihu optimization algorithm, dynamically update the search position of the optimization parameter set according to the parameter numerical performance of the initial reference parameter set and the historical optimization trajectory, and form a new heterogeneous graph structure optimization parameter set; S46. Perform fitness evaluation on the new heterogeneous graph structure optimization parameter set again, and feed back the evaluation results to the feedback knowledge base in real time. Continue to loop fitness evaluation, parameter update, and feedback knowledge base update until the preset optimization termination condition is met. Output the final optimized heterogeneous graph structure parameter set to form an optimized heterogeneous graph structure.

6. The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to claim 1 is characterized in that: The S5 specifically includes: S51. Based on the optimized heterogeneous graph structure, calculate the semantic connection strength change trend between data nodes in real time, identify and determine the core semantic connection path at the current moment, and output the data node set with the highest semantic connection strength change trend as the key node group in real time; S52. Analyze the semantic sensitivity of each key node group one by one, identify nodes with abnormal semantic fluctuations in real time, and expand from the abnormally fluctuating nodes to neighboring nodes, determine the semantic conflict diffusion path and the potential social risk impact range, and generate potential social risk warning information in spatial and semantic dimensions in real time; S53, obtaining the current evolution state of the semantic features of the key node group and its surrounding nodes in the heterogeneous graph structure in real time, determining the node semantic evolution trajectory at future consecutive moments based on the real-time semantic evolution speed and direction between the nodes, and generating event trend prediction information for each node; S54. Based on the current real-time node semantic evolution trajectory, determine the cross-node group semantic evolution path triggered between key node groups in a short time sequence, analyze the triggering conditions of the interaction of the semantic features of the node groups, output the associated trigger nodes, impact paths and evolution sequences between events, and form complete and continuous deduction information on the association relationship between events; S55. Establish a real-time monitoring mechanism for node semantic emergencies. By comparing the current node semantic state with the historical state change rate in real time, automatically capture sudden abnormal changes in node semantics, and automatically include sudden abnormal nodes in the early warning node set, and update potential social risk early warning information and event trend prediction information in real time.

7. The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to claim 1 is characterized in that: The S6 specifically includes: S61. Based on potential social risk warning information, event trend forecast information, and event correlation relationship deduction information, a multi-dimensional decision feature space containing risk level, event evolution rate, and probability of triggering related events is constructed in real time, and key decision factors of the current governance situation are dynamically determined; S62. Establish a multidimensional similarity evaluation matrix between current key decision-making factors and historical grassroots governance decision-making samples in real time. By calculating the multidimensional similarity between the key decision-making factor characteristics of the current situation and the characteristics of historical governance decision-making samples, automatically select the historical governance decision-making samples that have the highest degree of match with the current governance situation. S63. Analyze the governance effectiveness of each disposal measure in the sample after implementation based on the real-time screening of historical governance decision samples. Automatically generate an initial ranking of auxiliary decision measures based on the sample's governance effectiveness and the real-time similarity between the sample and the current governance context. S64. Real-time tracking of the dynamic evolution trend of key decision-making factors in the current situation. Based on the real-time change direction and rate of change of key decision-making factors, automatic dynamic adjustment of the initial auxiliary decision-making measures ranking, and real-time generation of dynamic auxiliary decision-making solutions that are consistent with the current governance situation. S65. Using the current dynamic decision-making support plan as input, perform real-time online simulation of the effectiveness of governance measures, automatically evaluate the real-time evolution trends of different governance measure combinations on key decision factors, and dynamically determine the optimal governance and disposal strategy sequence; S66. Push the optimal governance and disposal strategy sequence confirmed by online simulation to the grassroots governance decision-making terminal in real time, forming auxiliary decision-making plans and governance and disposal strategies that are dynamically updated in real time and adapt to rapid changes in governance situations.

8. The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to claim 7 is characterized in that: The S64 specifically includes: S641. Real-time extraction of multi-dimensional dynamic trend characteristics of key decision-making factors in the current governance context, including risk level change trends, event evolution rate change trends, and associated event trigger probability change trends, and establish trend evolution trajectories for each decision-making factor; S642. Analyze the trend evolution trajectory of each key decision factor one by one, determine the trend evolution inflection point of each decision factor and the possible time of the inflection point in real time, and automatically generate the stability and change sensitivity index of the trend state of each decision factor; S643. Based on the trend stability and change sensitivity indicators of key decision factors, a real-time matching evaluation model between decision-making support measures and key decision factors is constructed to evaluate in real time the degree of fit between each decision-making support measure and the current changing state of the decision factors. S644. Through the real-time matching evaluation model, determine the real-time trend fit score of each decision-making support measure in the current governance context, and dynamically quantify the immediate adaptability of the decision-making support measures; S645. Dynamically adjust the priority sequence of the initial decision-making support measures one by one based on the change in trend fit scores in real time, prioritizing the decision-making support measures with the highest trend fit scores at the front of the sequence, and forming a dynamic priority ranking of measures that adapts to the trend changes of the current decision-making factors in real time; S646. Monitor the changes in the trend evolution trajectory of key decision-making factors in real time, automatically update the trend evolution trajectory, trend stability and sensitivity indicators, and real-time matching evaluation models, and dynamically generate auxiliary decision-making plans that continuously match the current governance context.

9. The method for intelligent decision-making support for grassroots social governance based on multimodal fusion according to claim 1 is characterized in that: The S7 specifically includes: S71. Establish a semantic response model between feedback data of actual grassroots governance disposal strategies and expected governance scenarios in real time. Automatically identify semantic anomalies in governance effects by calculating the multi-dimensional semantic difference vectors between actual feedback results and governance scenarios. S72. For identified semantic anomalies, a governance effect difference space is constructed in real time based on the semantic difference vector. The core nodes and key connection paths that cause the governance effect to deviate from the expected target are determined through the real-time distribution characteristics of the semantic difference values ​​of each dimension in the difference space. S73. Conduct sensitivity analysis on the semantic difference value change trends of core nodes and key connection paths in real time. By quantifying the sensitivity of nodes and connection paths to governance effects in real time, identify nodes and connection relationships in the heterogeneous graph structure that urgently need to be updated. S74, using the core nodes and key connection paths determined by the sensitivity analysis as anchor points for dynamic optimization of the heterogeneous graph structure, calculating in real time the semantic feature offset between other nodes in the heterogeneous graph and the anchor nodes, and automatically determining the connection update area of ​​the heterogeneous graph structure based on the offset; S75. Perform a local dynamic reconstruction process of the heterogeneous graph structure connection relationship in real time based on the semantic feature offset, and automatically update the weight parameters of the feature fusion between nodes, so that the heterogeneous graph structure can dynamically adapt to the changes in the governance context reflected by the actual feedback results in real time; S76. Feedback the locally dynamically reconstructed heterogeneous graph structure and updated node fusion weight parameters into the Feihu optimization algorithm in real time, automatically triggering the Feihu optimization algorithm to start a new round of overall optimization process of the heterogeneous graph structure, and continuously and dynamically adjust the governance decision-making method.

10. A grassroots social governance intelligent decision support system based on multimodal fusion, which executes the grassroots social governance intelligent decision support method based on multimodal fusion as described in any one of claims 1 to 9, characterized in that: Includes the following modules: The data collection and preprocessing module is used to collect and preprocess text data, image data, video data, audio data, social media data, and IoT perception data in the field of grassroots social governance to generate structured and semi-structured data sets; Heterogeneous graph construction module, used to construct a heterogeneous graph structure with dynamic interactive relationships in multiple dimensions of space, time, semantics and themes based on governance topic categories and data types; The cross-modal feature fusion module is used to extract features within a single data modality based on heterogeneous graph structures and perform cross-modal feature interactive fusion to generate cross-modal dynamic correlation features; The heterogeneous graph optimization module uses the Feihu optimization algorithm to optimize the accuracy and real-time performance of grassroots governance decisions. It performs collaborative search and optimization on heterogeneous graph structures and cross-modal feature fusion parameters to obtain the optimized heterogeneous graph structure. The risk prediction and deduction module is used to generate potential social risk warning information, event trend prediction information, and event correlation relationship deduction information in real time based on the optimized heterogeneous graph structure; A decision-making support module is used to dynamically generate decision-making support plans and governance strategies that are highly compatible with the current governance context in real time based on potential social risk warning information, event trend forecast information, and information on the correlation between events. The feedback self-evolution module is used to input the feedback results of the actual implementation of auxiliary decision-making plans and disposal strategies into the Feihu optimization algorithm in real time, realize the dynamic update and continuous optimization of heterogeneous graph structures, and enable the grassroots social governance intelligent decision-making support system to continue to self-evolve.

Images