A Multimodal Collaborative Processing Method and System for Park Data Based on Dynamic Weight Fusion

Abstract

This invention provides a method and system for collaborative processing of multimodal park data based on dynamic weight fusion, belonging to the field of data processing technology. The method includes: obtaining J frequent risk type labels by frequently mining accident types from a historical risk event set of the park; establishing a park perception feature group by adaptive semantic parsing of the park's multimodal perception dataset; performing dynamic weight parsing to obtain dynamic weight configurations for each feature; performing dynamic fusion of risk contributions to obtain J risk contribution perception fusion matrices; performing multi-scale perturbation learning risk tracing to obtain a park risk tracing map; and performing multi-dimensional tracing drift parsing and correction to obtain an optimized risk tracing map. This invention solves the technical problem of traditional park management, which relies mainly on post-event handling and lacks proactive prediction capabilities based on multimodal data, leading to low efficiency in risk event response.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a method and system for collaborative processing of multimodal campus data based on dynamic weight fusion. Background Technology

[0002] Traditional park management typically relies on a "post-incident" approach to address risks and accidents. This means the park management system only responds and takes emergency measures after a risk event has occurred. This approach has significant limitations, especially in dealing with sudden accidents or complex environmental changes. It often fails to implement effective preventative or control measures in a timely manner, leading to escalated losses. Furthermore, park management often employs decision-making mechanisms based on a single data source, failing to fully integrate multimodal data or perform dynamic analysis and real-time prediction. This results in inefficient risk event response and missed critical windows for risk prevention. Summary of the Invention

[0003] This application provides a multimodal park data collaborative processing method and system based on dynamic weight fusion, which aims to solve the technical problem that traditional park management is mainly based on post-event handling and lacks the ability to proactively predict based on multimodal data, resulting in low efficiency in responding to risk events.

[0004] The first aspect disclosed in this application provides a multimodal park data collaborative processing method based on dynamic weight fusion. The method includes: obtaining J frequent risk type labels by frequently mining accident types from a historical risk event set of the park, where J is a positive integer greater than 1; establishing a park perception feature group by adaptive semantic parsing of the park's multimodal perception dataset; performing dynamic weight parsing on the park perception feature group based on the J frequent risk type labels according to a multi-level weight parsing mechanism to obtain dynamic weight configurations for each feature; dynamically fusing the risk contribution of the park perception feature group based on the J frequent risk type labels and the dynamic weight configurations for each feature to obtain J risk contribution perception fusion matrices; introducing a multi-level perturbation mechanism to perform multi-scale perturbation learning risk tracing on the J risk contribution perception fusion matrices in conjunction with the historical risk event set of the park to obtain a park risk tracing map; and performing multi-dimensional tracing drift parsing correction based on the park risk tracing map to obtain an optimized risk tracing map.

[0005] The second aspect of this application discloses a multimodal park data collaborative processing system based on dynamic weight fusion. This system is used in the aforementioned multimodal park data collaborative processing method based on dynamic weight fusion. The system includes: a frequent accident type mining module, used to obtain J frequent risk type labels by performing frequent accident type mining on a historical risk event set of the park, where J is a positive integer greater than 1; an adaptive semantic parsing module, used to establish a park perception feature group by performing adaptive semantic parsing on the park's multimodal perception dataset; and a dynamic weight parsing module, used to perform dynamic weight parsing based on the J frequent risk type labels according to a multi-level weight parsing mechanism. The system performs dynamic weight analysis on the park's perceived feature group to obtain the dynamic weight configuration of each feature; the risk contribution dynamic fusion module is used to perform risk contribution dynamic fusion on the park's perceived feature group based on the J frequent risk type labels and the dynamic weight configuration of each feature to obtain J risk contribution perception fusion matrices; the risk tracing module is used to introduce a multi-level perturbation mechanism and combine the park's historical risk event set to perform multi-scale perturbation learning risk tracing on the J risk contribution perception fusion matrices to obtain a park risk tracing map; the analysis and correction module is used to perform multi-dimensional tracing drift analysis and correction based on the park risk tracing map to obtain an optimized risk tracing map.

[0006] One or more technical solutions provided in this application have at least the following beneficial effects:

[0007] By frequently mining accident types from the park's historical risk event set, common risk types can be effectively screened, reducing data redundancy and ensuring that subsequent analysis focuses on the most influential risk factors, thus improving the efficiency and accuracy of subsequent analysis. Adaptive semantic parsing of the park's multimodal perception dataset allows for the extraction of key features from different data sources and their organic combination to form a complete perception feature group. This overcomes the heterogeneity between different data modalities and ensures synergistic effects of data at different levels. Introducing a multi-level weight parsing mechanism allows for dynamic adjustment of the weight configuration of each perception feature, making the fusion of perception data more intelligent and accurate. Based on dynamic weight configuration, and according to each frequent risk type label, the risk contribution of the park's perception feature group is dynamically fused to generate a risk contribution perception fusion. The matrix integrates the contribution values ​​of various perceptual features under different risk types, providing a comprehensive and weighted feature perspective for subsequent risk prediction. A multi-level perturbation mechanism is introduced, combining historical risk event sets for multi-scale perturbation learning. This mechanism simulates various possible scenarios and perturbation sources, enhancing the system's robustness in dealing with unknown risks and emergencies. Through perturbation learning, the evolution of different risk paths can be traced and predicted, constructing a risk tracing map for the park. After obtaining the park's risk tracing map, multi-dimensional tracing drift analysis and correction can dynamically optimize the map, correcting drift points and improving its accuracy. This process allows the system to continuously optimize the risk tracing map as the environment changes and new data is added, enabling park managers to quickly identify potential risks and take targeted prevention and control measures.

[0008] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0009] Figure 1 A schematic diagram of the multimodal campus data collaborative processing method based on dynamic weight fusion provided in this application embodiment.

[0010] Figure 2 This is a schematic diagram of the structure of a multimodal campus data collaborative processing system based on dynamic weight fusion, provided in an embodiment of this application.

[0011] Figure labeling: 10 for frequent accident type mining module, 20 for adaptive semantic parsing module, 30 for dynamic weight parsing module, 40 for risk contribution dynamic fusion module, 50 for risk tracing module, and 60 for parsing correction module. Detailed Implementation

[0012] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0013] Example 1, as Figure 1 As shown in the embodiments of this application, a multimodal campus data collaborative processing method based on dynamic weight fusion is provided, the method comprising:

[0014] By frequently mining accident types from the historical risk event set of the park, J frequent risk type labels are obtained, where J is a positive integer greater than 1.

[0015] Furthermore, by frequently mining accident types from the historical risk event set of the park, J frequent risk type labels were obtained, including:

[0016] Accident type identification is performed based on the historical risk event set of the park to obtain an accident type label set; the trigger frequency of each accident type label in the accident type label set is evaluated based on the historical risk event set of the park to obtain multiple type trigger frequency coefficients; based on the multiple type trigger frequency coefficients, the trigger frequency of the accident type label set is optimized according to the type trigger frequency threshold to generate the J frequent risk type labels.

[0017] The system collects a historical risk event set for the park, including information on all types of risk events that have occurred in the past, such as safety accidents, natural disasters, and equipment failures. These events are presented in time-series format, including information such as the time, type, location, and scope of impact. The system analyzes this historical risk event set to identify all accident types, such as fires, equipment failures, and power outages. Based on the identified accident types, each event is assigned a corresponding accident type label, generating an accident type label set, where each label represents a risk type.

[0018] For each accident type label, calculate its frequency of occurrence in the historical risk event set. That is, count the number of times each type appears in the historical event set. The type trigger frequency coefficient of each accident type label represents the frequency of the label in historical events, which is the number of times the type label appears divided by the total number of historical risk events.

[0019] Set a frequency threshold for type triggers, which is the minimum value of a frequency coefficient. Only accident type tags with a frequency coefficient greater than or equal to this threshold are considered frequent risk types. The threshold can be set according to actual needs, such as based on historical data and the requirements of the actual scenario. Compare multiple type trigger frequency coefficients with the set type trigger frequency threshold, and filter out accident type tags with a frequency coefficient greater than or equal to the threshold to obtain J frequent risk type tags. These tags play a key role in the park's risk management because they represent risk types with a high probability of occurrence or a significant impact on the park.

[0020] By performing adaptive semantic parsing on the multimodal perception dataset of the park, a perception feature group of the park is established.

[0021] The park's multimodal sensing dataset is provided by various sensors and monitoring devices, including visual sensors, audio sensors, environmental sensors, positioning devices, and other data sources. Adaptive semantic parsing extracts useful features from this raw sensing data and fuses them according to different risk types and data sources. This process includes data cleaning and preprocessing, feature extraction and abstraction, and semantic parsing. After adaptive semantic parsing, all extracted sensing features are organized into a park sensing feature group, which contains key features under various sensing modalities in the park, such as temperature, humidity, smoke concentration, visual features, and audio features.

[0022] Based on the J frequent risk type labels, dynamic weight analysis is performed on the park perception feature group according to the multi-level weight analysis mechanism to obtain the dynamic weight configuration of each feature.

[0023] The multi-level weighting analysis mechanism dynamically assigns weights to each feature in the park's perceived feature group based on multiple weighting factors. Different risk types have different requirements for the sensitivity, credibility, and temporal changes of perceived features. Therefore, the weighting analysis mechanism needs to evaluate features at multiple levels and assign weights based on different standards. The multi-level weighting factors include risk association weights, perceived credibility weights, and temporal dynamic weights. Dynamic weighting analysis includes risk association evaluation, credibility evaluation, and temporal sensitivity evaluation. Based on the above evaluation results, combined with the setting of multi-level weighting factors, dynamic weights are assigned to each feature in the park's perceived feature group. These weights reflect the importance of each feature under different risk types, ultimately obtaining the dynamic weight configuration for each perceived feature.

[0024] Based on the J frequent risk type labels, the risk contribution of the park perception feature group is dynamically fused according to the dynamic weight configuration of each feature to obtain J risk contribution perception fusion matrices.

[0025] The contribution of each perceived feature to a specific risk type is calculated using its weight and the actual value of the perceived feature. For example, for the fire risk type, the perceived features of temperature and smoke concentration are multiplied by real-time data according to their dynamic weights to obtain the fire risk contribution value. For each frequent risk type label, each feature in the perceived feature group is weighted according to the relevant dynamic weight configuration of the perceived features to calculate the risk contribution of the feature.

[0026] By fusing the risk contributions of all perceived features, methods such as weighted average and weighted sum can be used to combine the contributions of all features in the perceived feature group to obtain a comprehensive risk contribution perception fusion matrix for each frequent risk type. This matrix reflects the contribution of all perceived features under each frequent risk type and indicates which features play a decisive role in a specific risk scenario.

[0027] A multi-level perturbation mechanism is introduced, and multi-scale perturbation learning risk tracing is performed on the J risk contribution perception fusion matrices in combination with the historical risk event set of the park to obtain the park risk tracing map.

[0028] The multi-level perturbation mechanism perturbs the sensed data at different scales, including low-frequency, mid-frequency, and high-frequency, to simulate data changes under different conditions. These perturbations simulate the random fluctuations and uncertainties of factors such as the park environment, equipment operation, and the sensed system. To better simulate real-world scenarios, the perturbation mechanism is trained using a set of historical risk events in the park. By reviewing historical risk events and the actual environment of the park, the impact of various perturbation patterns on park security risks can be assessed.

[0029] Multi-scale perturbation is applied to J risk contribution perception fusion matrices to generate multiple perturbed risk contribution matrices. Based on these perturbed matrices, machine learning methods are used to train a model, further improving the predictive ability of risks. After multi-scale perturbation learning, a risk tracing map of the park is established, which presents the correlation between different risk types and park perception data, perturbation data, and historical events.

[0030] Based on the risk tracing map of the park, multi-dimensional tracing drift analysis and correction are performed to obtain an optimized risk tracing map.

[0031] Tracing drift refers to the temporal and spatial drift analysis based on multiple data sources during risk tracing. In this step, by analyzing changes in each risk path in the park's risk tracing map, influencing factors such as data acquisition-related drift, environmental-related drift, and topological-related drift are identified. Based on the analyzed types of drift, combined with information such as the park's specific environmental prediction trends and topological change trends, the park's risk tracing map is adjusted. This correction process aims to eliminate errors caused by data drift, ensuring more accurate risk prediction. Ultimately, an optimized risk tracing map is obtained. This optimized map more accurately reflects the park's risk evolution process, providing more reliable support for risk management, emergency response, and early warning systems.

[0032] Furthermore, based on the J frequent risk type labels, dynamic weight parsing is performed on the park perception feature group according to a multi-level weight parsing mechanism to obtain the dynamic weight configuration of each feature, including:

[0033] The multi-level weighting analysis mechanism is activated. This mechanism includes multi-level weighting factors, which include risk association weights, perceived credibility weights, and time-series dynamic weights. A risk association distribution of perceived features is obtained by evaluating the association between the J frequent risk type labels and the park's perceived feature group. A perceived credibility evaluation distribution of perceived features is obtained by evaluating the perceived credibility of the park's perceived feature group. A time-change sensitivity evaluation distribution of perceived features is obtained by evaluating the perceived sensitivity of the park's perceived feature group. Based on the multi-level weighting factors, multi-level weight allocation is performed on the park's perceived feature group according to the perceived feature risk association distribution, the perceived feature credibility evaluation distribution, and the perceived feature sensitivity distribution, to obtain the dynamic weight configuration for each feature.

[0034] The multi-level weighting analysis mechanism refers to a comprehensive analysis method based on multi-level weighting factors, which can assign different weight values ​​to each sensing feature. Among them, the risk correlation weight measures the correlation between the sensing feature and the park's risks, and assesses the impact of the sensing feature on the early warning capability of different types of risks; the sensing credibility weight assesses the accuracy and reliability of the sensing data, that is, whether the sensing feature is trustworthy. For example, factors such as data source, collection method, and sensor quality will affect its credibility; the time-series dynamic weight assesses the timeliness and changing trend of the sensing feature, analyzes the dynamics and time-series correlation of the feature changing over time. For parks under real-time monitoring, certain sensing features are more important in specific time periods.

[0035] By analyzing the correlation between J frequent risk type labels and the park's perception feature group, the correlation degree between each perception feature and each risk type is calculated. This correlation evaluation can be carried out using methods such as correlation analysis. Based on the analysis results, a perception feature risk correlation distribution is generated. This distribution shows the strength of the relationship between each perception feature and different risk types, reflecting the early warning capabilities of different perception features for various risks.

[0036] Perception reliability assessment evaluates the quality and reliability of perception data within the park. This process assesses the reliability of each perception feature based on factors such as the data source, sensor quality, and environmental conditions during data acquisition. Error analysis, including measurement errors and data loss, as well as sensor calibration and historical verification methods, can be used. A reliability assessment algorithm generates a reliability evaluation value for each perception feature, ultimately forming a perception feature reliability evaluation distribution that reflects the reliability of each feature.

[0037] Time-varying sensitivity assessment evaluates the impact of changes in a park's sensory characteristics over time. Through time-series analysis of these characteristics, such as time series analysis and dynamic trend analysis, it assesses the importance of each sensory characteristic across different time periods. Based on the analysis results, a time-varying sensitivity distribution is generated, indicating the changing trends of sensory characteristics at different points in time or across different time periods, and their sensitivity to risk.

[0038] Based on the previously generated risk association distribution, credibility evaluation distribution, and sensitivity evaluation distribution of perceived features, and combined with risk association weights, credibility weights, and time-series dynamic weights, a multi-level weight allocation is performed on each perceived feature. Through the weight allocation process, a dynamic weight configuration is generated for each perceived feature. The dynamic weight configuration of each feature reflects the importance of each perceived feature under a specific risk type and can be dynamically adjusted with changes in time and environment.

[0039] Furthermore, a multi-level perturbation mechanism is introduced, and multi-scale perturbation learning risk tracing is performed on the J risk contribution perception fusion matrices based on the historical risk event set of the park to obtain a park risk tracing map, including:

[0040] The historical risk event set of the park is classified according to the J frequent risk type labels to obtain J frequent risk event areas; risk tracing accident tree organization is performed on the J frequent risk event areas to obtain J frequent risk tracing sample sets; multi-scale perturbation federated aggregation learning is performed on the J frequent risk tracing sample sets according to the multi-level perturbation mechanism to build J frequent risk prediction and tracing channels; the J risk contribution perception fusion matrices are input into the J frequent risk prediction and tracing channels to obtain J frequent risk prediction and tracing paths; the J frequent risk prediction and tracing paths are organized to generate the park risk tracing map.

[0041] Based on J frequent risk type labels, the set of historical risk events in the park is classified, and each type of historical risk event is assigned to the corresponding frequent risk event area. That is, each frequent risk type label corresponds to a set of historical risk events. For example, events with the historical label "fire" are classified into the fire frequent risk event area, and events with the historical label "equipment failure" are classified into the equipment failure frequent risk event area.

[0042] Fault tree analysis is a systematic risk analysis method used to identify potential causes of system failures. Here, risk events within J frequent risk event zones are organized to construct risk tracing fault trees. Each fault tree starts from the occurrence of a risk event and traces it step-by-step to its possible root cause. Through the organization of fault trees, key factors and potential risk patterns of the event can be revealed. For all events in each frequent risk event zone, fault tree analysis is applied to generate J frequent risk tracing sample sets. Each sample set corresponds to a risk type and includes its related tracing information, such as the root cause of the accident and the risk chain.

[0043] The multi-level perturbation mechanism is a method to improve the generalization ability of a model by simulating perturbations. Here, multi-scale perturbation learning is performed on J frequent risk tracing sample sets to simulate various possible changes and uncertainties. The multi-scale perturbations include low-frequency perturbations, mid-frequency perturbations and high-frequency perturbations. These perturbations simulate the impact of risk factors at different levels on the prediction results, ensuring that the model can adapt to different situations and data changes.

[0044] Federated learning is a distributed machine learning method that allows model training on distributed data without centralizing the data. In this scenario, federated learning enables independent training of models on different frequent risk tracing sample sets, followed by aggregating the results of each model, thus avoiding data leakage and improving computational efficiency. Each sample set is trained through a multi-level perturbation mechanism to generate a risk prediction model. Finally, federated aggregation learning combines the outputs of all sub-models to obtain J global frequent risk prediction tracing channels, each corresponding to a specific risk type, for subsequent risk prediction and analysis.

[0045] J risk contribution perception fusion matrices are input into J established frequent risk prediction and tracing channels. Through the prediction model of the channels, the specific impact of each perception feature on different frequent risk types is calculated. Each channel generates a corresponding frequent risk prediction and tracing path based on the perception feature data and risk type labels in the matrix. These paths demonstrate how to trace and predict park risks based on perception features.

[0046] By organizing J frequent risk prediction and tracing paths into a comprehensive park risk tracing map, it can clearly show the occurrence chain and trend of various risks in the park, providing a basis for risk management and early warning in the park.

[0047] Furthermore, based on the aforementioned multi-level perturbation mechanism, multi-scale perturbation federated aggregation learning is performed on the J frequent risk tracing sample sets to construct J frequent risk prediction and tracing channels, including:

[0048] Extract the j-th frequent risk tracing sample set from the J frequent risk tracing sample sets, where j is a positive integer, 1≤j≤J; activate the multi-level perturbation mechanism, which includes low-frequency perturbation scale, mid-frequency perturbation scale, and high-frequency perturbation scale; inject multi-scale perturbation into the j-th frequent risk tracing sample set according to the multi-level perturbation mechanism to obtain a first risk tracing perturbation sample set, a second risk tracing perturbation sample set, and a third risk tracing perturbation sample set; train the j-th risk prediction tracing base model based on the j-th frequent risk tracing sample set; train the first risk prediction tracing model, the second risk prediction tracing model, and the third risk prediction tracing perturbation sample set based on the first risk tracing perturbation sample set, the second risk prediction tracing perturbation sample set, and the third risk prediction tracing perturbation sample set; perform federated aggregation learning between the j-th risk prediction tracing base model and the first risk prediction tracing model, the second risk prediction tracing model, and the third risk prediction tracing model to generate the j-th frequent risk prediction tracing channel.

[0049] Extract the j-th frequent risk tracing sample set from the J frequent risk tracing sample sets, where j represents any risk type. The j-th frequent risk tracing sample set contains event data related to the j-th risk type and is ready to proceed to the next step of perturbation learning processing.

[0050] Low-frequency disturbances target long-term trend changes, such as seasonal variations and economic cycle fluctuations. These disturbances affect long-term data patterns and correspond to slower changes, such as the long-term effects of environmental factors or equipment aging. Mid-frequency disturbances involve periodic fluctuations, such as monthly or quarterly variations. These disturbances simulate the impact of periodic risk factors on events, such as seasonal changes or periodic maintenance issues. High-frequency disturbances target short-term, sudden changes, such as momentary equipment failures or human error. These disturbances primarily simulate the impact of sudden risk factors, which change rapidly and require timely response and intervention.

[0051] Through a multi-level perturbation mechanism, perturbations of different scales are introduced into the j-th frequent risk tracing sample set to simulate potential risk events under different conditions. Specifically, the first risk tracing perturbation sample set, obtained by perturbing the j-th frequent risk tracing sample set with a low-frequency perturbation scale, reflects long-term trend changes, such as equipment aging and seasonal weather changes; the second risk tracing perturbation sample set, obtained by perturbing the j-th frequent risk tracing sample set with a mid-frequency perturbation scale, reflects changes in periodic factors, such as annual maintenance and fluctuations in holiday traffic; and the third risk tracing perturbation sample set, obtained by perturbing the j-th frequent risk tracing sample set with a high-frequency perturbation scale, simulates short-term sudden risk events, such as power outages and instantaneous equipment failures.

[0052] The basic risk prediction model is trained using the j-th frequent risk tracing sample set, i.e., the j-th risk prediction tracing base model. This base model mainly relies on the original sample set, i.e., undisturbed data, for training. The trained base model can be based on neural networks, decision trees, random forests, etc. This base model is mainly used to predict the j-th frequent risk type, such as predicting the probability of occurrence and the scope of impact of this type of risk event.

[0053] Similar to the j-th frequent risk tracing sample set, for each disturbance sample set, an independent risk prediction model is trained, resulting in the first risk prediction tracing model, the second risk prediction tracing model, and the third risk prediction tracing model. By training these three models, risks can be predicted under different disturbance scales. Each model adapts to different disturbance scenarios, thereby enhancing the comprehensiveness and adaptability of the prediction.

[0054] Federated learning is used to aggregate the base model for predicting and tracing the j-th risk with the first, second, and third risk prediction and tracing models. Federated learning combines the prediction results of multiple models to obtain a more accurate and robust prediction model without having to store all the data in one place. Finally, a channel for predicting and tracing the j-th frequent risk is generated. This channel contains multi-perspective predictions of the j-th frequent risk type and can integrate risk information in different scenarios to provide more accurate risk predictions.

[0055] Furthermore, based on the aforementioned risk tracing map of the park, multi-dimensional tracing drift analysis and correction are performed to obtain an optimized risk tracing map, including:

[0056] The multimodal perception dataset is analyzed for anomalies in the data acquisition scenarios to obtain anomalies in each data acquisition scenario. Based on the anomalies in each data acquisition scenario, the tracing drift analysis is performed on each frequent risk prediction tracing path in the park risk tracing map to obtain the acquisition-related tracing drift distribution. Based on the park environment prediction trend, the tracing drift identification is performed on each frequent risk prediction tracing path to obtain the environment-related tracing drift distribution. Based on the park topology dataset, the tracing drift identification is performed on each frequent risk prediction tracing path to obtain the topology-related tracing drift distribution. Based on the acquisition-related tracing drift distribution, the environment-related tracing drift distribution, and the topology-related tracing drift distribution, the park risk tracing map is adaptively corrected to generate the optimized risk tracing map.

[0057] In the analysis of multimodal sensing datasets, anomaly analysis is performed on each data acquisition scenario. This involves analyzing the raw data to identify potential abnormal events or unexpected data behaviors, including sensor malfunctions, data acquisition errors, missing data, or abnormal fluctuations. Anomaly analysis identifies anomalies in each data acquisition scenario, pinpointing which sensing systems or data acquisition events exhibit abnormalities. This allows subsequent analysis and prediction to avoid interference from these anomalous data.

[0058] Based on the anomalies obtained from various data collection scenarios, drift analysis is performed to trace the frequent risk prediction paths within the park's risk tracing map, i.e., the prediction paths of park risk events. The aim is to detect and analyze whether risk prediction paths have drifted or changed under abnormal data collection scenarios. For example, some data collection anomalies may cause deviations in risk event prediction paths, affecting subsequent risk assessments. Through analysis, the drift distribution of data collection associations is obtained, i.e., the relationship between data collection anomalies and risk prediction path drift. This helps identify prediction biases caused by collection anomalies.

[0059] Environmental prediction trends in the industrial park include weather changes, seasonal variations, and population flows, all of which often impact the park's safety. For each frequent risk prediction tracing path, analysis is conducted to determine whether it is affected by environmental changes and whether drift exists. For example, severe weather may increase the risk exposure of certain facilities, thus affecting the accuracy of the prediction path. Through analysis, the distribution of environmental correlation tracing drift is obtained, i.e., the impact of environmental changes on risk prediction paths. This information indicates the role of environmental factors in risk evolution, allowing for dynamic adjustments.

[0060] A campus topology dataset describes the spatial and logical relationships between various physical or virtual elements within a campus, such as buildings, facilities, equipment, and network nodes. Topology analysis helps identify how the layout and structure of facilities within the campus affect the propagation and development of risk events. Based on the campus topology dataset, drift identification is performed on various frequent risk prediction tracing paths. By analyzing the topological relationships within the campus, such as the connection methods between devices, the location of emergency exits, and the coverage of surveillance cameras, it is analyzed whether these factors have a drift impact on risk prediction paths. If the campus topology changes, such as facility relocation or network topology adjustments, it may lead to changes in risk propagation paths, thereby affecting the accuracy of predictions. Through topology analysis, the distribution of topology-related tracing drift is obtained, that is, the impact of the campus topology on risk path drift. This information helps optimize the layout of campus facilities and risk prediction strategies.

[0061] By comprehensively collecting and tracking drift distributions related to the environment, topology, and other factors, the risk tracing map of the park is optimized. This includes correcting predicted paths, reassessing risk points, and redefining sensitive areas. The core of adaptive correction is to dynamically adjust the risk prediction model based on the impact of new environments, data collection, and topology, making the map more consistent with the actual situation of the park. After adaptive correction, an optimized risk tracing map is generated.

[0062] Furthermore, based on the park topology dataset, tracing drift identification is performed on each frequent risk prediction tracing path to obtain the topological association tracing drift distribution, including:

[0063] Based on the park topology dataset, a park topology model is constructed; based on the park topology model, topological association analysis is performed on each frequent risk prediction tracing path to obtain a tracing path topology graph; based on the tracing path topology graph, propagation trend deduction is performed on each frequent risk prediction tracing path according to the park topology model to obtain the topological propagation deduction results for each risk; based on the topological propagation deduction results for each risk, drift identification is performed on each frequent risk prediction tracing path to obtain the topological association tracing drift distribution.

[0064] A topological model of the park is constructed based on the park topology dataset. This model represents the relationships between elements in the park in the form of a graph structure. The construction of the park topology model can be achieved through graph theory methods, where each element is regarded as a node in the graph, and the relationships between them are regarded as edges.

[0065] Using the constructed park topology model, topological association analysis is performed on each frequent risk prediction and tracing path. The analysis results can reveal the relationship between each frequent risk prediction and tracing path and each node in the park. This information constitutes the tracing path topology diagram, which is a graphical structure that reflects the risk propagation path and mutual influence between various facilities and areas in the park.

[0066] Based on the tracing path topology diagram and the park topology model, the propagation trend of each frequent risk prediction tracing path is simulated. This step mainly simulates how risk events propagate from one area or node to other areas or nodes. The simulation process considers multiple factors, such as facility connectivity, personnel flow, and environmental conditions, simulating various possible risk propagation paths. The simulation results show the propagation trend of each risk prediction path under different conditions. These trends not only reveal how risks spread within the park but also predict potential risk hotspots, helping the park to conduct effective resource allocation and emergency response planning.

[0067] Based on the risk topology propagation simulation results, drift identification is performed on each frequent risk prediction and tracing path. During the risk propagation process, changes in the park, such as equipment failure, personnel intervention, and changes in the external environment, may cause the propagation path to drift, meaning the path or intensity of risk propagation changes. Drift identification compares the simulation results with the actual situation to identify whether there are deviations from expectations on the propagation path. Through drift identification, the topology association tracing drift distribution is obtained, that is, the drift status of each frequent risk prediction and tracing path.

[0068] Furthermore, adaptive semantic parsing is performed on the multimodal perception dataset of the park, including:

[0069] Multimodal monitoring data of the park is collected to obtain a multimodal park dataset; multidimensional preprocessing is performed on the multimodal park dataset to generate the multimodal perception dataset, wherein the multidimensional preprocessing includes data cleaning, time synchronization and feature normalization.

[0070] Multimodal monitoring data refers to a collection of data from different sources and sensors within the park, including environmental data, video data, and sound data, ultimately forming a comprehensive and rich multimodal park dataset. This dataset not only covers various fields within the park but can also be combined with various data sources for multi-angle monitoring and analysis.

[0071] Data cleaning refers to denoising and correcting raw data, handling missing, outlier, or erroneous data, and filling in missing values ​​using interpolation, mean imputation, or other methods. Different sensors collect data at different time intervals, so it's necessary to synchronize time data from different sources. Time synchronization ensures that all data sources are analyzed on the same timeline, avoiding data bias caused by time misalignment. Feature normalization adjusts features at different scales to the same standard range, using methods such as min-max normalization or Z-score normalization. After completing the above preprocessing, a multimodal perception dataset is finally obtained, providing a higher-quality data foundation for subsequent model training and risk analysis.

[0072] Furthermore, based on the risk tracing optimization map, a park early warning instruction is generated, and dynamic park management is performed based on the park early warning instruction.

[0073] Based on the risk tracing and optimization map, early warning instructions are generated for the park. These instructions automatically trigger warnings based on potential risk points identified in the map, trend projections, and drift analysis results. The warnings include specific risk types, locations, and urgency levels, helping managers respond quickly. Dynamic park management is based on real-time response measures taken according to these early warning instructions, including facility scheduling, resource allocation, and environmental adjustments, ensuring the park's safety and sustainability.

[0074] Example 2, based on the same inventive concept as the multimodal campus data collaborative processing method based on dynamic weight fusion in the previous examples, such as... Figure 2 As shown in the figure, this application provides a multimodal campus data collaborative processing system based on dynamic weight fusion, the system comprising:

[0075] The system comprises the following modules: Accident Type Frequent Mining Module 10, which mines accident types frequently from the historical risk event set of the park to obtain J frequent risk type labels, where J is a positive integer greater than 1; Adaptive Semantic Parsing Module 20, which performs adaptive semantic parsing on the multimodal perception dataset of the park to establish a park perception feature group; Dynamic Weight Parsing Module 30, which performs dynamic weight parsing on the park perception feature group based on the J frequent risk type labels according to a multi-level weight parsing mechanism to obtain dynamic weight configurations for each feature; Risk Contribution Dynamic Fusion Module 40, which performs dynamic fusion of risk contributions on the park perception feature group based on the J frequent risk type labels and the dynamic weight configurations for each feature to obtain J risk contribution perception fusion matrices; Risk Tracing Module 50, which introduces a multi-level perturbation mechanism and performs multi-scale perturbation learning risk tracing on the J risk contribution perception fusion matrices in conjunction with the historical risk event set of the park to obtain a park risk tracing map; and Parsing Correction Module 60, which performs multi-dimensional tracing drift parsing correction based on the park risk tracing map to obtain an optimized risk tracing map.

[0076] Furthermore, the accident type frequent detection module 10 is used to perform the following operation steps:

[0077] Accident type identification is performed based on the historical risk event set of the park to obtain an accident type label set; the trigger frequency of each accident type label in the accident type label set is evaluated based on the historical risk event set of the park to obtain multiple type trigger frequency coefficients; based on the multiple type trigger frequency coefficients, the trigger frequency of the accident type label set is optimized according to the type trigger frequency threshold to generate the J frequent risk type labels.

[0078] Furthermore, the dynamic weight parsing module 30 is used to perform the following operation steps:

[0079] The multi-level weighting analysis mechanism is activated. This mechanism includes multi-level weighting factors, which include risk association weights, perceived credibility weights, and time-series dynamic weights. A risk association distribution of perceived features is obtained by evaluating the association between the J frequent risk type labels and the park's perceived feature group. A perceived credibility evaluation distribution of perceived features is obtained by evaluating the perceived credibility of the park's perceived feature group. A time-change sensitivity evaluation distribution of perceived features is obtained by evaluating the perceived sensitivity of the park's perceived feature group. Based on the multi-level weighting factors, multi-level weight allocation is performed on the park's perceived feature group according to the perceived feature risk association distribution, the perceived feature credibility evaluation distribution, and the perceived feature sensitivity distribution, to obtain the dynamic weight configuration for each feature.

[0080] Furthermore, the risk tracing module 50 is used to perform the following operation steps:

[0081] The historical risk event set of the park is classified according to the J frequent risk type labels to obtain J frequent risk event areas; risk tracing accident tree organization is performed on the J frequent risk event areas to obtain J frequent risk tracing sample sets; multi-scale perturbation federated aggregation learning is performed on the J frequent risk tracing sample sets according to the multi-level perturbation mechanism to build J frequent risk prediction and tracing channels; the J risk contribution perception fusion matrices are input into the J frequent risk prediction and tracing channels to obtain J frequent risk prediction and tracing paths; the J frequent risk prediction and tracing paths are organized to generate the park risk tracing map.

[0082] Furthermore, the risk tracing module 50 is used to perform the following operation steps:

[0083] Extract the j-th frequent risk tracing sample set from the J frequent risk tracing sample sets, where j is a positive integer, 1≤j≤J; activate the multi-level perturbation mechanism, which includes low-frequency perturbation scale, mid-frequency perturbation scale, and high-frequency perturbation scale; inject multi-scale perturbation into the j-th frequent risk tracing sample set according to the multi-level perturbation mechanism to obtain a first risk tracing perturbation sample set, a second risk tracing perturbation sample set, and a third risk tracing perturbation sample set; train the j-th risk prediction tracing base model based on the j-th frequent risk tracing sample set; train the first risk prediction tracing model, the second risk prediction tracing model, and the third risk prediction tracing perturbation sample set based on the first risk tracing perturbation sample set, the second risk prediction tracing perturbation sample set, and the third risk prediction tracing perturbation sample set; perform federated aggregation learning between the j-th risk prediction tracing base model and the first risk prediction tracing model, the second risk prediction tracing model, and the third risk prediction tracing model to generate the j-th frequent risk prediction tracing channel.

[0084] Furthermore, the parsing correction module 60 is used to perform the following operation steps:

[0085] The multimodal perception dataset is analyzed for scene anomalies to obtain anomalies in each data collection scene. Based on these anomalies, the frequent risk prediction tracing paths within the park risk tracing map are analyzed for tracing drift to obtain the collection-related tracing drift distribution. Based on the park environment prediction trend, the frequent risk prediction tracing paths are identified for tracing drift to obtain the environment-related tracing drift distribution. Based on the park topology dataset, the frequent risk prediction tracing paths are identified for tracing drift to obtain the topology-related tracing drift distribution. The park risk tracing map is adaptively corrected based on the collection-related tracing drift distribution, the environment-related tracing drift distribution, and the topology-related tracing drift distribution to generate the optimized risk tracing map.

[0086] Furthermore, the parsing correction module 60 is used to perform the following operation steps:

[0087] Based on the park topology dataset, a park topology model is constructed; based on the park topology model, topological association analysis is performed on each frequent risk prediction tracing path to obtain a tracing path topology graph; based on the tracing path topology graph, propagation trend deduction is performed on each frequent risk prediction tracing path according to the park topology model to obtain the topological propagation deduction results for each risk; based on the topological propagation deduction results for each risk, drift identification is performed on each frequent risk prediction tracing path to obtain the topological association tracing drift distribution.

[0088] Furthermore, the adaptive semantic parsing module 20 is used to perform the following operational steps:

[0089] Multimodal monitoring data of the park is collected to obtain a multimodal park dataset; multidimensional preprocessing is performed on the multimodal park dataset to generate the multimodal perception dataset, wherein the multidimensional preprocessing includes data cleaning, time synchronization and feature normalization.

[0090] Furthermore, based on the risk tracing optimization map, a park early warning instruction is generated, and dynamic park management is performed based on the park early warning instruction.

[0091] Through the foregoing detailed description of the multimodal campus data collaborative processing method based on dynamic weight fusion, those skilled in the art can clearly understand the multimodal campus data collaborative processing system based on dynamic weight fusion in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and relevant parts can be referred to the method section.

[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for collaborative processing of multi-modal park data based on dynamic weight fusion, characterized in that, The method includes: By frequently mining accident types from the historical risk event set of the park, J frequent risk type labels are obtained, where J is a positive integer greater than 1; By performing adaptive semantic parsing on the multimodal perception dataset of the park, a perception feature group of the park is established; Based on the J frequent risk type labels, dynamic weight analysis is performed on the park perception feature group according to the multi-level weight analysis mechanism to obtain the dynamic weight configuration of each feature. Based on the J frequent risk type labels, the risk contribution dynamic fusion of the park perception feature group is performed according to the dynamic weight configuration of each feature to obtain J risk contribution perception fusion matrices. The aforementioned dynamic risk contribution fusion refers to, for each frequent risk type label, dynamically configuring the relevant perceived features according to their weights, weighting each feature in the perceived feature group, calculating the risk contribution of each feature, fusing the risk contributions of all perceived features, and using weighted average and weighted sum methods to combine the contributions of all features in the perceived feature group to obtain a comprehensive risk contribution perception fusion matrix for each frequent risk type. This matrix reflects the contribution of all perceived features under each frequent risk type. A multi-level perturbation mechanism is introduced, and multi-scale perturbation learning risk tracing is performed on the J risk contribution perception fusion matrices in combination with the historical risk event set of the park to obtain the park risk tracing map. Based on the aforementioned risk tracing map of the park, multi-dimensional tracing drift analysis and correction are performed to obtain an optimized risk tracing map; The step of performing multi-dimensional tracing drift analysis and correction based on the park's risk tracing map to obtain an optimized risk tracing map includes: Anomaly analysis of the acquisition scene is performed on the multimodal perception dataset to obtain anomalies in each data acquisition scene; Based on the anomalies in each data collection scenario, the tracing drift analysis is performed on each frequent risk prediction tracing path in the park risk tracing map to obtain the collection-related tracing drift distribution. Based on the predicted trends of the park environment, the traceability drift of each frequent risk prediction traceability path is identified to obtain the environmental association traceability drift distribution. Based on the park topology dataset, traceability drift identification is performed on each frequent risk prediction traceability path to obtain the topology-related traceability drift distribution. The risk tracing map of the park is adaptively corrected based on the drift distribution of the collected correlation, the drift distribution of the environmental correlation, and the drift distribution of the topological correlation to generate the optimized risk tracing map.

2. The multimodal campus data collaborative processing method based on dynamic weight fusion as described in claim 1, characterized in that, By frequently mining accident types from the historical risk event set of the park, J frequent risk type labels were obtained, including: Accident type identification is performed based on the historical risk event set of the park to obtain an accident type label set; Based on the historical risk event set of the park, the trigger frequency of each accident type label in the accident type label set is evaluated to obtain multiple trigger frequency coefficients for each type. Based on the multiple trigger frequency coefficients, the accident type label set is optimized for trigger frequency according to the type trigger frequency threshold to generate the J frequent risk type labels.

3. The multimodal park data collaborative processing method based on dynamic weight fusion as described in claim 1, characterized in that, Based on the J frequent risk type labels, dynamic weight parsing is performed on the park perception feature group according to a multi-level weight parsing mechanism to obtain the dynamic weight configuration of each feature, including: Activate the multi-level weight parsing mechanism, which includes multi-level weight factors, including risk association weight, perceived credibility weight, and time-series dynamic weight. By evaluating the correlation between the J frequent risk type labels and the park's perception feature group, the risk correlation distribution of perception features is obtained; By performing a perception credibility evaluation on the perception feature group of the park, the perception feature credibility evaluation distribution is obtained; By performing a time-varying sensitivity evaluation on the sensory feature group of the park, the sensory feature sensitivity evaluation distribution is obtained; Based on the multi-level weighting factors, the perception feature group of the park is weighted in a multi-level manner according to the perception feature risk association distribution, the perception feature credibility evaluation distribution and the perception feature sensitivity evaluation distribution, so as to obtain the dynamic weight configuration of each feature.

4. The multimodal campus data collaborative processing method based on dynamic weight fusion as described in claim 1, characterized in that, A multi-level perturbation mechanism is introduced, and multi-scale perturbation learning risk tracing is performed on the J risk contribution perception fusion matrices based on the historical risk event set of the park to obtain a park risk tracing map, including: The historical risk event set of the park is classified according to the J frequent risk type labels to obtain J frequent risk event areas; Risk tracing fault tree analysis is performed on the J frequent risk event areas to obtain J frequent risk tracing sample sets; Based on the multi-level perturbation mechanism, multi-scale perturbation federated aggregation learning is performed on the J frequent risk tracing sample sets to build J frequent risk prediction and tracing channels. Input the J risk contribution perception fusion matrices into the J frequent risk prediction and tracing channels to obtain J frequent risk prediction and tracing paths; The J frequent risk prediction and tracing paths are organized to generate the park risk tracing map.

5. The multimodal campus data collaborative processing method based on dynamic weight fusion as described in claim 4, characterized in that, Based on the multi-level perturbation mechanism, multi-scale perturbation federated aggregation learning is performed on the J frequent risk tracing sample sets to build J frequent risk prediction and tracing channels, including: Extract the j-th frequent risk tracing sample set based on the J frequent risk tracing sample sets, where j is a positive integer, 1≤j≤J; Activate the multi-level perturbation mechanism, which includes a low-frequency perturbation scale, a mid-frequency perturbation scale, and a high-frequency perturbation scale; According to the multi-level perturbation mechanism, the j-th frequent risk tracing sample set is subjected to multi-scale perturbation injection to obtain the first risk tracing perturbation sample set, the second risk tracing perturbation sample set, and the third risk tracing perturbation sample set; Based on the j-th frequent risk tracing sample set, train the j-th risk prediction tracing base model; Based on the first risk tracing disturbance sample set, the second risk tracing disturbance sample set, and the third risk tracing disturbance sample set, train the first risk prediction and tracing model, the second risk prediction and tracing model, and the third risk prediction and tracing model; The j-th risk prediction and tracing base model is federated and aggregated with the risk prediction and tracing first model, the risk prediction and tracing second model, and the risk prediction and tracing third model to generate the j-th frequent risk prediction and tracing channel.

6. The multimodal campus data collaborative processing method based on dynamic weight fusion as described in claim 1, characterized in that, Based on the park topology dataset, traceability drift is identified for each frequent risk prediction traceability path to obtain the topological association traceability drift distribution, including: Based on the aforementioned park topology dataset, construct a park topology model; Based on the park topology model, the topological association of each frequent risk prediction and tracing path is analyzed to obtain the tracing path topology diagram. Based on the tracing path topology diagram, the propagation trend of each frequent risk prediction tracing path is deduced according to the park topology model to obtain the propagation deduction results of each risk topology. Based on the risk topology propagation results, drift identification is performed on the frequent risk prediction and tracing paths to obtain the topology-related tracing drift distribution.

7. The multimodal campus data collaborative processing method based on dynamic weight fusion as described in claim 1, characterized in that, Adaptive semantic parsing was performed on the multimodal perception dataset of the park, including: Collect multimodal monitoring data of the park to obtain a multimodal park dataset; The multimodal campus dataset is preprocessed in multiple dimensions to generate the multimodal perception dataset. The multimodal preprocessing includes data cleaning, time synchronization, and feature normalization.

8. The multimodal campus data collaborative processing method based on dynamic weight fusion as described in claim 1, characterized in that, Based on the risk tracing optimization map, a park early warning instruction is generated, and dynamic park management is executed according to the park early warning instruction.

9. A multimodal campus data collaborative processing system based on dynamic weight fusion, characterized in that, The system is used to implement the multimodal campus data collaborative processing method based on dynamic weight fusion as described in any one of claims 1-8, the system comprising: The accident type frequent mining module is used to obtain J frequent risk type labels by frequently mining accident types from the historical risk event set of the park, where J is a positive integer greater than 1; The adaptive semantic parsing module is used to establish a cluster of perceptual features of the park by performing adaptive semantic parsing on the multimodal perception dataset of the park. The dynamic weight parsing module is used to perform dynamic weight parsing on the park perception feature group based on the J frequent risk type labels and according to the multi-level weight parsing mechanism to obtain the dynamic weight configuration of each feature. The risk contribution dynamic fusion module is used to perform risk contribution dynamic fusion on the park perception feature group based on the J frequent risk type labels and according to the dynamic weight configuration of each feature, to obtain J risk contribution perception fusion matrices. The risk tracing module is used to introduce a multi-level perturbation mechanism, combine the historical risk event set of the park with the J risk contribution perception fusion matrices to perform multi-scale perturbation learning risk tracing, and obtain the park risk tracing map; The analysis and correction module is used to perform multi-dimensional traceability drift analysis and correction based on the risk traceability map of the park, and obtain an optimized risk traceability map.

Images