Park multistage intelligent reasoning and early warning system based on multi-modal knowledge graph

Abstract

The invention provides an intelligent early warning system fusing Internet of Things sensing data and a domain knowledge graph, aiming at the problems of data islands, high false alarm rate, response lag and the like of a traditional park early warning system, and is suitable for park safety prevention and control in industries such as chemical industry, logistics, manufacturing and the like. The knowledge graph is an ideal tool for modeling connection between objective objects in the real world, the data island problem can be effectively solved by constructing the knowledge graph oriented to the smart park safety management field and fusing an intelligent reasoning algorithm, and the accuracy and timeliness of park risk early warning are remarkably improved. Specifically, a whole set of pre-warning system is designed from bottom to top in three aspects of multi-modal knowledge graph modeling, a three-level pre-warning inference engine and a self-adaptive optimization mechanism, and the park pre-warning requirements which meet current intellectualization and manpower cost saving are constructed. The multi-modal knowledge graph relates to six types of ontology concepts, comprises different data types, and comprises an equipment topological relation, environmental parameter association, an emergency plan, risk analysis, attack behavior simulation and an asset attribute model. The third-level early warning reasoning comprises rule reasoning, sub-graph matching reasoning and link prediction reasoning. The self-adaptive optimization technology aims at constructing a feedback learning mechanism, incorporating each early warning processing result into a knowledge graph, and continuously optimizing the object relation weight. In an early warning analog simulation experiment, the scheme of the invention realizes the effects of reducing the false alarm rate by 42% and improving the emergency response speed by 60%, and the feasibility and effectiveness of the scheme are proved.

Description

Technical Field

[0001] The present invention belongs to the field of smart park security management and timely development, and relates to a multi-level intelligent early warning system that integrates Internet of Things perception data and domain knowledge graphs, which is suitable for security protection of industrial parks such as smart logistics and smart manufacturing. Background Art

[0002] The Knowledge Graph is a technical framework that describes real-world entities and their relationships in a structured form. Its core goal is to enable machines to understand and apply human knowledge through semantic networks. Since Google first proposed the concept of "Knowledge Graph" in 2012, this technology has become the core infrastructure for artificial intelligence, big data analysis, and semantic search. Triples (head entity-relationship-tail entity), also known as relational facts, are the basic building blocks of the knowledge graph. For example, (Beijing, is the capital, China) represents the relational fact that Beijing is the capital of China. The core technologies of the knowledge graph include knowledge acquisition and construction, knowledge storage and query, knowledge representation learning, and knowledge reasoning and application. Compared with ordinary text data, knowledge graphs are more suitable for discovering implicit associations between entity objects. Therefore, they are widely used in intelligent search, intelligent question-answering systems, security protection, medical health, and intelligent recommendations.

[0003] In terms of park early warning, the current mainstream park early warning system uses a four-tier classification system aligned with international standards. Level IV Blue Alert corresponds to general risk events, placing the system in basic monitoring mode and requiring only routine data collection and recording. Typical scenarios include fluctuations in equipment parameters and minor deviations in environmental indicators. Level III Yellow Alert indicates a more severe risk and requires enhanced monitoring frequency (e.g., reducing sensor sampling intervals by 50%) and triggering a primary response plan. This is often seen in the event of a warning of equipment failure or localized environmental anomalies. Level II Orange Alert signifies a severe threat, automatically activating cross-departmental coordination mechanisms and pre-deploying emergency resources. This often occurs in the event of a regional hazardous source leak or major equipment failure. Level I Red Alert, the highest level, triggers the park-wide emergency response process, initiating mandatory evacuation orders and requests for external assistance simultaneously. This system is suitable for extreme events such as earthquake precursors and hazardous chemical explosions. Some advanced parks use a five-tier system, adding a "white monitoring level" before the Blue Alert for more refined control.

[0004] The existing park early warning system has two main problems: one is the serious data island phenomenon, and the other is the lag in the algorithm model, which leads to low accuracy and timeliness of the early warning system. To this end, this invention proposes a bottom-up systematic solution.

[0005] First, in terms of data island management, six categories of ontology concepts were designed, including equipment topology relationships, environmental parameter associations, emergency plans, risk analysis, attack behavior simulation, and asset attribute models, and a multimodal knowledge graph was constructed based on IoT sensor data. Multimodal Knowledge Graph (MKG) is an evolved form of traditional knowledge graphs. It is a semantic network constructed by integrating multi-source heterogeneous data such as text, images, audio, and video. In particular, in the research of the present invention, sensor data is neither audio nor image or video data, but physical quantity data. For example, when handling a pipeline corridor leakage accident, the light sensor detects abnormal vibration and triggers an alarm. This vibration data belongs to physical quantity data. These data are included in the multimodal knowledge graph in the patent of this invention. Secondly, in terms of algorithm model lag, the patent of this invention will propose a three-level early warning intelligent reasoning model. The specific design ideas are as follows: First-level reasoning: rule-based engine processing to identify threshold alarms (e.g., temperature > 80°C triggers an alarm); Second-level reasoning: through subgraph matching, such as using graph neural networks (GNNs) to achieve dynamic mapping of sensor data and knowledge nodes, such as associating reasoning between the vibration frequency of the pipe gallery and the standard value of structural stress to discover cross-system risk transmission paths (e.g., chemical leakage → ventilation system failure → pollution diffusion in areas where people gather); Third-level reasoning: using time-series knowledge graphs to predict risk trends in the next 30 minutes, improving accuracy and timeliness. Finally, based on multimodal data and a three-level warning intelligent reasoning model, an adaptive optimization mechanism is introduced to further reduce manual intervention and reduce labor costs. A feedback learning closed loop is constructed, and the results of each warning disposal are added as new edges to the graph to continuously optimize the ontological relationship weights.

[0006] To sum up, in order to improve the accuracy and timeliness of existing park safety warnings, the patent of this invention takes "multimodal data construction → multi-level warning intelligent reasoning → adaptive optimization mechanism" as the main line, and designs a complete park warning system from the bottom up. In simulation experiments, compared with traditional methods such as relying solely on sensor alarms, it has achieved the ideal effect of reducing the false alarm rate by 42% and increasing the emergency response speed by 60%. Summary of the Invention

[0007] In order to overcome the defects and shortcomings of data islands and algorithm lags in the existing park early warning system, the present invention proposes a park multi-level intelligent reasoning early warning system based on multimodal knowledge graph.

[0008] The technical solution adopted by the present invention is a bottom-up systematic solution of "multimodal data construction → multi-level warning intelligent reasoning → adaptive optimization mechanism", which has the following features:

[0009] Step 1: Data collection and modality alignment. First, data is collected from device topology, environmental parameters, emergency response plans, risk analysis, attack behavior simulation, and asset attribute models. Then, multimodal data is converted into structured data through cross-modal feature mapping.

[0010] Step 2: Ontology modeling and knowledge extraction, including six-dimensional ontology architecture design and multimodal entity recognition, establishing the management relationship between the various entity objects of the early warning system, and preliminarily completing the construction of the multimodal knowledge graph;

[0011] Step 3: Dynamic association and incremental update: perform cross-type association, calculate similar entities through semantic similarity, establish association relationships, and design an incremental graph update strategy. This step further improves the goal of building a multimodal knowledge graph.

[0012] Step 4: Based on the constructed multimodal knowledge graph, design the technical architecture of the three-level warning reasoning engine to initially achieve the goal of improving the accuracy and timeliness of warnings;

[0013] Step 5: Build an adaptive early warning optimization system to reduce labor costs and automatically update the ontology relationship weights through a feedback learning mechanism to further improve the performance of the early warning system.

[0014] Furthermore, the step 1 specifically includes the following steps:

[0015] Step 1-1): Acquire multi-source data, including device topology data: Obtain device connection diagrams from the SCADA system and build a physical hierarchy based on CAD drawings; Environmental parameters: Collect time-series data such as temperature, humidity, and vibration through IoT sensors, and synchronize video surveillance streams as visual aids; Emergency Response Plan: Structured analysis of national emergency standard documents (such as GB / T 29639) to extract response processes and responsible parties.

[0016] Steps 1-2): Cross-modal feature mapping uses the CLIP model to align text plans with the 3D equipment model. For example, a vector association is established between the text instructions for "Fire Pump Startup Procedure" and a screenshot of the equipment operation interface. Statistical charts in the risk analysis report are converted into structured data through OCR.

[0017] Furthermore, the step 2 specifically includes the following steps:

[0018] Step 2-1): Design the six-dimensional ontology architecture. First, establish some ontology schema layers. The multimodal knowledge graph is constructed based on the ontology concepts and enriches the knowledge graph strength data from top to bottom. The model is as follows:

[0019]

[0020] Step 2-2): Multimodal entity recognition. Text modality: Use the BERT-BiLSTM model to extract entities such as "PLC model: S7-1200" from the device manual. Visual modality: YOLOv7 identifies the color of device status indicators in surveillance videos and maps them to the asset attribute library. Time series modality: LSTM detects abnormal segments in sensor data and marks them as risk event nodes.

[0021] Furthermore, the step 3 specifically includes the following steps:

[0022] Step 3-1): Dynamic association, cross-ontology association rule association type example technology implementation equipment - ambient temperature exceeds the standard trigger cooling system startup timing pattern matching attack - asset vulnerability CVE-2025-001 associated PLC firmware version threat intelligence map traversal plan - risk chemical leakage plan matching LD50 toxicity data semantic similarity calculation;

[0023] Step 3-2): Incremental graph updates: Device log streams are received in real time via Kafka. Graph Neural Networks (GNNs) are used to dynamically update topological relationships. When a new device is detected, an asset audit is automatically triggered. Conditional judgment: When the GNN output indicates a new device probability > 0.9, a REST API is automatically called to initiate the audit process. Audit content includes basic information verification (MAC address whitelist comparison); security configuration checks (open ports and default credentials scanning); and compliance verification (firmware version compliance with IEC 62443).

[0024] Furthermore, the specific steps of step 4 are as follows:

[0025] Step 4-1): First-level reasoning: rule-driven threshold alerting. Implementation principle: The Drools rule engine processes structured data, with 382 pre-set industry-standard rules (e.g., "pressure vessel temperature > 120°C for > 3 minutes" triggers an emergency shutdown). A dynamic threshold adjustment algorithm automatically adjusts the alarm threshold based on equipment aging (e.g., the temperature threshold decreases by 2.3°C for every additional year of service).

[0026] Step 4-2): Secondary reasoning: Knowledge graph association analysis. Subgraph matching technology: Construct a campus knowledge graph containing 170,000 entities and 530,000 relationships, using the GraphSAGE algorithm to achieve:

[0027] #Example: Discovering associated paths for leak incidents

[0028] match(n1:Sensor)-[r:NEARBY]->(n2:Pipeline)

[0029] where n1.value>threshold

[0030] return n1,r,n2

[0031] Step 4-3): Three-level reasoning: Time series situation forecasting. Hybrid modeling approach: Integrating LSTM neural networks with TransE graph embedding technology. Input layer: Historical sensor data (60-minute time window); Hidden layer: 256 LSTM units + knowledge node attention mechanism; Output layer: Risk probability distribution for the next 30 minutes (MAE = 0.08).

[0032] Furthermore, the step 5 specifically includes the following steps:

[0033] Step 5-1): Feedback learning closed-loop design, establish the "warning event-->handling record-->map update-->model retraining-->new warning generation" data flow architecture.

[0034] Step 5-2: Use the Bayesian probabilistic graphical model to dynamically update the relationship weights. For example, after a certain type of gas sensor gives three consecutive false alarms, the confidence of its association rule drops from 0.85 to 0.72.

[0035] In general, the above technical solution conceived by the present invention has the following technical features and effects compared with the prior art:

[0036] 1. Make full use of data from various types of park sensor equipment to build a multimodal knowledge graph in the field of early warning, and be able to achieve dynamic knowledge growth, ensure the validity and timeliness of data, effectively solve the traditional data island problem, and lay the foundation for the subsequent early warning reasoning algorithm design.

[0037] 2. Implementation of the three-level early warning reasoning engine technical architecture to achieve the goal of precise early warning reasoning in a progressive manner.

[0038] 3. Establish an adaptive early warning optimization scheme, and dynamically update the weight relationship through feedback learning closed-loop design and Bayesian probabilistic graphical model, further strengthening the early warning system capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 :Flowchart of the early warning system

[0040] Figure 2 : Diagram of the working principle of three-level warning intelligent reasoning DETAILED DESCRIPTION

[0041] In order to facilitate those skilled in the art to understand and implement the present invention, the following is a diagram of the present invention with reference to the accompanying drawings ( Figure 1 and Figure 2 ) and Examples The present invention is described in further detail:

[0042] First, we built a data source classification system and established a data collection matrix covering 7 categories and 32 subcategories:

[0043] IoT device data: DAS distributed optical fiber (sampling rate 1kHz), infrared thermal imager (accuracy

[0044] ±0.5℃)

[0045] Business system data: ERP work order records, BIM model component library (LOD400 standard)

[0046] Environmental perception data: weather station (wind speed and direction, PM2.5), drone inspection images (resolution 4K@30fps)

[0047] Text knowledge base: 28 industry standard documents including GB / T 50493-2019, historical accident reports (unstructured PDF)

[0048] Secondly, heterogeneous data governance and cross-modal feature mapping were performed. A dedicated ETL tool chain was developed to implement data governance. The CLIP model was used to align text plans with equipment 3D models. For example, a vector association was established between the text instructions for "Fire Pump Startup Procedure" and screenshots of the equipment operation interface. Statistical charts in the risk analysis report were converted into structured data through OCR. Some code snippets for this implementation are as follows:

[0049]

[0050] Step 2: Ontology modeling and knowledge extraction

[0051] First, a six-dimensional ontology architecture is constructed. The domain ontology is constructed using the Protégé tool to define the core ontology class, which is defined as follows:

[0052]

[0053]

[0054] Secondly, multimodal entity recognition is performed. Entity recognition is performed on different data types, such as structured data: relational data is mapped to RDF using Spark SQL; text data: a BERT-BiLSTM-CRF model (F1 = 0.83) is used to extract triplets from accident reports; and visual data: YOLOv5 + ResNet152 is used to identify equipment nameplate information.

[0055] Step 3: Dynamic association and incremental update of knowledge

[0056] First, cross-ontology association rules are generated and a rule template library is constructed. Twelve common rule templates are predefined (e.g., "If the distance between a Class A device and a Class B environmental sensor is less than 5m, then establish a monitors relationship"). Online rule mining uses the Frequent Pattern Outlier Factor (FPOF) algorithm to discover new association patterns from real-time data streams. For example, when the frequency of the combination "vibration frequency > 35Hz and temperature > 80°C" in a certain campus data suddenly increases, a rule for inferring device health status is automatically generated.

[0057] Secondly, we designed an incremental topology update subgraph: We used the ChangeProp algorithm to identify graph change areas. When a new "T-101 tank leak" event was added, we only started updating the adjacent nodes (valves, pipelines, sensors) within a 3-hop range. Dynamic representation learning was also implemented: We designed the TGAT (Temporal Graph Attention) model to process temporal edges.

[0058] Step 4: Three-level warning intelligent reasoning engine

[0059] Level 1 reasoning: Rule-driven threshold alerts. Operational steps: Using the Drools rule engine to process structured data, 382 pre-set industry-standard rules are used (e.g., "pressure vessel temperature > 120°C for > 3 minutes" triggers an emergency shutdown). Dynamic threshold adjustment algorithm: Automatically adjusts alarm thresholds based on equipment aging (e.g., the temperature threshold decreases by 2.3°C for every additional year of service).

[0060] Secondary reasoning: Knowledge graph association analysis and subgraph matching technology: Construct a campus knowledge graph containing 170,000 entities and 530,000 relationships, and apply the GraphSAGE algorithm.

[0061] Three-level reasoning: performs time series situation forecasting. This system integrates LSTM neural networks and TransE graph embedding technology: Input layer: historical sensor data (60-minute time window), Hidden layer: 256 LSTM units + knowledge node attention mechanism, Output layer: risk probability distribution for the next 30 minutes (MAE = 0.08).

[0062] Step 5: Adaptive warning optimization

[0063] First, the multi-source feedback collection layer within the feedback learning closed-loop design process is implemented as follows: mobile annotation: On-site personnel use the app to annotate warning results with a triplet of "valid / false positive / missed negative," along with associated action records (such as the type of extinguishing agent actually used); automated verification: comparing the SCADA system's final status with the warning prediction results to generate quantitative error metrics (such as a temperature prediction deviation of ±2.3°C); knowledge graph iteration: dynamically adjusting the confidence level of relationships like "vibration-leakage" (initial 0.75 → 0.82 after feedback), and adding new action rules (such as the empirical formula "foam dosage = area × 0.35").

[0064] Secondly, the Bayesian dynamic weight adjustment process is specifically implemented as follows: prior distribution setting, with the initial relationship weight derived from the historical data Beta distribution (α = 15, β = 5), defining the weight update trigger condition (such as three consecutive feedback errors greater than the threshold), and online Bayesian parameter update. The likelihood function uses the Bernoulli distribution to model the probability of successful treatment. Posterior sampling uses the MCMC algorithm to update the weight distribution in real time.

[0065] It should be understood that, under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A multi-level intelligent early warning management system for industrial parks. Its innovation lies in the use of knowledge graph technology to achieve multi-source data fusion and intelligent reasoning. Its steps are as follows: Step 1: Data collection and modality alignment. First, data is collected from device topology, environmental parameters, emergency response plans, risk analysis, attack behavior simulation, and asset attribute models. Then, multimodal data is converted into structured data through cross-modal feature mapping. Step 2: Ontology modeling and knowledge extraction, including six-dimensional ontology architecture design and multimodal entity recognition, establishing the management relationship between the various entity objects of the early warning system, and preliminarily completing the construction of the multimodal knowledge graph; Step 3: Dynamic association and incremental update: perform cross-type association, calculate similar entities through semantic similarity, establish association relationships, and design an incremental graph update strategy. This step further improves the goal of building a multimodal knowledge graph. Step 4: Based on the constructed multimodal knowledge graph, design the technical architecture of the three-level warning reasoning engine to initially achieve the goal of improving the accuracy and timeliness of warnings; Step 5: Build an adaptive early warning optimization system to reduce labor costs and automatically update the ontology relationship weights through a feedback learning mechanism to further improve the performance of the early warning system.

2. The multi-level intelligent early warning management system for industrial parks according to claim 1, wherein the specific steps include: The step 1 specifically includes the following steps: Step 1-1): Acquire multi-source data, including device topology data: Obtain device connection diagrams from the SCADA system and build a physical hierarchy based on CAD drawings; Environmental parameters: Collect time-series data such as temperature, humidity, and vibration through IoT sensors, and synchronize video surveillance streams as visual aids; Emergency Response Plans: Structured analysis of national emergency standard documents (such as GB / T 29639) to extract response processes and responsible parties; Steps 1-2): Cross-modal feature mapping uses the CLIP model to align text plans with the 3D equipment model. For example, a vector association is established between the text instructions for "Fire Pump Startup Procedure" and a screenshot of the equipment operation interface. Statistical charts in the risk analysis report are converted into structured data through OCR.

3. The multi-level intelligent early warning management system for industrial parks according to claim 1, wherein the specific steps include: The step 2 specifically includes the following steps: Step 2-1): Design the six-dimensional ontology architecture. First, establish some ontology schema layers. The multimodal knowledge graph is constructed based on the ontology concepts and enriches the knowledge graph strength data from top to bottom. The model is as follows: #Example of ontology class definition class IndustrialOntology: device_topology = Graph() #Device connection diagram env_params = TimeSeriesDB() #Environmental parameter time series database emergency_plans = NLP_Module() #Semantic analysis of emergency plans risk_models = BayesianNetwork() # Risk probability graph attack_patterns = ThreatIntelAPI()#attack pattern library asset_attributes = Neo4j() #Asset attribute map Step 2-2): Multimodal entity recognition. Text modality: Use the BERT-BiLSTM model to extract entities such as "PLC model: S7-1200" from the device manual. Visual modality: YOLOv7 identifies the color of device status indicators in surveillance videos and maps them to the asset attribute library. Time series modality: LSTM detects abnormal segments in sensor data and marks them as risk event nodes.

4. The multi-level intelligent early warning management system for industrial parks according to claim 1, wherein the specific steps include: The step 3 specifically includes the following steps: Step 3-1): Dynamic association, cross-ontology association rule association type example technology implementation equipment - ambient temperature exceeds the standard trigger cooling system startup timing pattern matching attack - asset vulnerability CVE-2025-001 associated PLC firmware version threat intelligence map traversal plan - risk chemical leakage plan matching LD50 toxicity data semantic similarity calculation; Step 3-2): Incremental graph update: Receive device log streams in real time through Kafka, use graph neural networks (GNN) to dynamically update topology relationships, and automatically trigger the asset audit process when a new device is detected. Conditional judgment: When the GNN output probability of a new device is >0.9, the REST API is automatically called to start the audit process; Audit content: Basic information verification: Compare MAC address whitelist; Security configuration check: Scan open ports and default credentials; Compliance verification: Check whether the firmware version complies with the IEC62443 standard.

5. The multi-level intelligent early warning management system for industrial parks according to claim 1, wherein the specific steps include: The specific steps of step 4 are as follows: Step 4-1): First-level reasoning: rule-driven threshold alarms. Implementation principle: Using the Drools rule engine to process structured data, 382 pre-set industry-standard rules are used (e.g., "pressure vessel temperature > 120°C for > 3 minutes" triggers an emergency shutdown). Dynamic threshold adjustment algorithm: Automatically adjusts alarm thresholds based on equipment aging factors (e.g., for every additional year of service, the temperature threshold decreases by 2.3°C). Step 4-2): Secondary reasoning: Knowledge graph association analysis; subgraph matching technology: Construct a campus knowledge graph containing 170,000 entities and 530,000 relationships, using the GraphSAGE algorithm: #Example: Discovering associated paths for leak incidents match(n1:Sensor)-[r:NEARBY]->(n2:Pipeline) where n1.value>threshold return n1,r,n2 Step 4-3): Three-level reasoning: Time series situation prediction; Hybrid modeling approach: Fusion of LSTM neural network and TransE graph embedding technology; Input layer: sensor historical data (time window 60 minutes); hidden layer: 256 LSTM units + knowledge node attention mechanism; output layer: risk probability distribution in the next 30 minutes (MAE = 0.08).

6. The multi-level intelligent early warning management system for industrial parks according to claim 1, wherein the specific steps include: The step 5 specifically includes the following steps: Step 5-1): Design a closed-loop feedback learning process and establish a data flow architecture of "warning event --> disposal record --> graph update --> model retraining --> new warning generation"; Step 5-2: Use the Bayesian probabilistic graphical model to dynamically update the relationship weights. For example, after a certain type of gas sensor gives three consecutive false alarms, the confidence of its association rule drops from 0.85 to 0.72.

Images