Safety monitoring method, system, equipment and medium

Abstract

The invention discloses a safety monitoring method. The method comprises the following steps: firstly, determining a set comprising three types of monitoring modes of equipment, personnel and environment; performing real-time reasoning on the original data through an AI model deployed at the edge to generate standardized modal features; by taking a fixed time window as a period, compulsively fusing the multi-modal features in the same period into a data record with a fixed structure, and keeping the structure complete even if the data is missing; and finally, based on the structured record, risk coupling analysis is carried out by using a dynamic probability graph model. According to the method, the problem of analysis fragmentation caused by different formats and frequencies of the multi-source heterogeneous data is solved through forced structured fusion, and stable input is provided for subsequent analysis; and then the probabilistic graph model is utilized to process the input, and the complex dependency relationship among equipment, personnel and environment states can be explicitly quantified, so that the accurate identification and real-time evaluation of the cross-dimension coupling risk which is difficult to identify by a traditional method are realized, and the active early warning capability of industrial safety management is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent safety monitoring, in particular to a device-personnel-environment trinity safety monitoring method, system, device and medium based on artificial intelligence, which is especially suitable for real-time sensing, coupling analysis and active early warning of safety risks in complex industrial environments such as construction sites, metallurgical plants, chemical parks and urban underground comprehensive pipe galleries. BACKGROUND

[0002] In the field of industrial safety management, equipment operation anomalies, non-standard personnel behavior and environmental condition deterioration are three main risk sources that are interrelated and dynamically influenced. The traditional safety monitoring methods have the following outstanding problems:

[0003] Firstly, data fragmentation and islandization. Equipment state data (such as vibration and temperature parameters from SCADA systems), personnel behavior data (such as video monitoring and positioning information) and environmental parameter data (such as temperature and humidity, and harmful gas concentration) are usually derived from independent information systems and sensor networks. These data are not uniform in format, collection frequency and time reference, lack effective correlation analysis framework, form "data islands", and make it difficult to fully and accurately grasp the overall safety situation.

[0004] Secondly, risk identification lags behind and the false alarm rate is high. Existing methods mostly rely on manual inspection or simple alarms based on single-point threshold values, and cannot perform real-time reasoning on the coupling risks across the three dimensions of equipment, personnel and environment. For example, a single slight vibration of equipment may not constitute a high risk, but when it occurs simultaneously with personnel approaching the equipment area and environmental temperature and humidity anomalies, it constitutes a typical high-risk combination. However, traditional systems lack fusion analysis capabilities and are unable to identify such complex risk patterns, resulting in delayed response or a large number of false alarms caused by single indicator fluctuations.

[0005] Thirdly, the risk prediction capability is weak. Most monitoring systems are passive response type and only alarm after an anomaly occurs. Due to the inability to deeply integrate and dynamically model the real-time states of equipment, personnel and environment, the system cannot predict the trend of future risk evolution based on the current multi-dimensional situation, missing the valuable time window for active prevention.

[0006] Finally, real-time performance cannot meet the requirements of high-risk scenarios. Centralized intelligent analysis solutions based on the cloud have strong computing power, but face problems such as network transmission delay, high bandwidth pressure, cloud processing queue waiting, etc. The cycle from event occurrence to generation of warning is long, making it difficult to meet the stringent requirements of immediate response to risks at the level of seconds or even milliseconds (such as mechanical collision and personnel entering dangerous areas).

[0007] Therefore, the industry urgently needs a set of active safety monitoring solutions that can effectively integrate heterogeneous data of equipment, personnel, and environment, implement low-latency and high-reliability intelligent analysis on the edge side, and identify and predict complex coupled risks in real time. SUMMARY

[0008] The present application aims to solve the problems of cross-modal risk identification difficulty, response lag, high false alarm rate, and lack of risk prediction ability caused by data silos in the prior art, and provides an intelligent solution that can efficiently and standardizedly fuse three types of risk source data of equipment, personnel, and environment, and implement real-time inference and coupled risk warning on the edge side. To achieve the above purpose, the present application adopts the following technical solutions.

[0009] In a first aspect, the embodiments of the present application provide a safety monitoring method, comprising:

[0010] determining a modal set for a specific monitoring object; the modal set includes equipment monitoring modal, personnel monitoring modal, and environment monitoring modal;

[0011] performing real-time inference on the collected original data corresponding to the modal set by deploying an inference node on the edge side, to generate standardized modal features;

[0012] fusing all modal features generated within the same time window and belonging to the same monitoring object to generate a structured window-level fusion record; the window-level fusion record contains fixed fields for representing the equipment, personnel, and environment states of the monitoring object within the time window;

[0013] based on the window-level fusion record, performing risk coupling analysis through a dynamic risk probability graph model, and outputting the coupling risk identification and risk probability of the monitoring object.

[0014] Further, the step of generating a structured window-level fusion record comprises:

[0015] For each type of modal in the modal set, if there is a valid modal feature corresponding to the type of modal within the current time window, then the representative feature value of the type of modal in the current window is extracted or calculated from the valid modal feature based on a preset rule;

[0016] if there is no valid modal feature of a certain type of modal within the current time window, then a missing state marker is generated for the type of modal;

[0017] fill the representative feature value or the missing state marker into the corresponding fixed field of the window-level fusion record according to a preset order.

[0018] Further, the step of generating standardized modal features by the inference node deployed at the edge side comprises:

[0019] training special artificial intelligence models corresponding to the equipment, personnel and environment modalities respectively based on historical data in the cloud;

[0020] performing lightweight processing on the trained special artificial intelligence models to obtain lightweight inference models;

[0021] deploying the lightweight inference models to the inference node at the edge side, loading and running the inference node, performing inference on real-time streaming raw data, and outputting the standardized modal features.

[0022] Further, the dynamic risk probability graph model is a Bayesian network model, the nodes of which include evidence nodes corresponding to equipment states, personnel behaviors and environment parameters, and risk nodes representing different coupling risk types; the step of performing risk coupling analysis comprises:

[0023] inputting the modal feature values in the window-level fusion record as evidence to the corresponding evidence nodes of the Bayesian network model;

[0024] dynamically updating the posterior probabilities of each risk node according to the evidence and the preset node conditional probability table in the model;

[0025] determining the risk represented by the risk node whose posterior probability exceeds a preset threshold as the coupling risk identifier, and outputting the posterior probability thereof as the risk probability.

[0026] Further, it further comprises:

[0027] when the risk probability exceeds a warning threshold, generating warning information containing risk cause tracing information according to the risk node type triggering the warning and the corresponding evidence node state; the warning information is used to drive the corresponding safety response operation.

[0028] Further, it further comprises:

[0029] collecting historical warning events and their corresponding window-level fusion records to construct a labeled risk sample database;

[0030] using the risk sample database to periodically incrementally train and optimize the parameters of the dynamic risk probability graph model.

[0031] Further, the standardized modal features of the personnel monitoring modality are human body skeleton key point time sequence feature generated by a pose estimation algorithm based on video image data;

[0032] The dynamic risk probability graph model analyzes the spatio-temporal correlation between the human skeleton key point time sequence track features and the equipment operation features to identify the collision risk of personnel misentering the equipment motion envelope.

[0033] In a second aspect, the embodiments of the present application provide a safety monitoring system capable of implementing the safety monitoring method of any of the preceding aspects, comprising:

[0034] a modal set determination module configured to determine a modal set for a specific monitoring object; the modal set comprising an equipment monitoring modal, a personnel monitoring modal, and an environment monitoring modal;

[0035] an edge inference module deployed at an edge side and configured to perform real-time inference on raw data of the modal set collected to generate standardized modal features;

[0036] a fusion record generation module configured to fuse all modal features of a same monitoring object generated within a same time window to generate a structured window-level fusion record in a preset time window; the window-level fusion record comprising fixed fields for representing equipment, personnel, and environment states of the monitoring object within the time window;

[0037] a risk analysis module configured to perform risk coupling analysis on the window-level fusion record based on a dynamic risk probability graph model to output a coupling risk identification and a risk probability of the monitoring object.

[0038] In a third aspect, the embodiments of the present application provide an electronic device, comprising: one or more processors;

[0039] a memory configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the steps in the safety monitoring method of any of the preceding aspects.

[0040] In a fourth aspect, the embodiments of the present application provide a computer readable medium having a computer program stored thereon; when the computer program is executed by a processor, the steps in the safety monitoring method of any of the preceding aspects can be implemented.

[0041] The application discloses a safety monitoring method, which first determines a set of three types of monitoring modalities including equipment, personnel and environment; through the AI model deployed at the edge, the original data is inferred in real time to generate standardized modality features; with a fixed time window as the period, the multi-modal features of the same period are forcibly fused into a data record with a fixed structure, and the structure is complete even if the data is missing; finally, based on this structured record, a dynamic probabilistic graph model is used for risk coupling analysis. The application solves the problem of fragmented analysis of multi-source heterogeneous data caused by different formats and frequencies by forced structured fusion, providing stable input for subsequent analysis; then, the probabilistic graph model is used to process the input, which can explicitly quantify the complex dependence between equipment, personnel and environment, thereby realizing accurate identification and real-time evaluation of cross-dimensional coupling risks that are difficult to identify by traditional methods, and improving the proactive warning capability of industrial safety management. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A core process block diagram of an equipment-personnel-environment trinity safety monitoring method based on artificial intelligence provided by an embodiment of the application;

[0043] Figure 2 A module structure schematic diagram of an equipment-personnel-environment trinity safety monitoring system based on artificial intelligence provided by an embodiment of the application;

[0044] Figure 3 A structural block diagram of an electronic device provided by an embodiment of the application. DETAILED DESCRIPTION

[0045] In order for those skilled in the art to better understand the technical solutions of the present application, the following describes exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to help understanding, which should be considered only as exemplary. Therefore, those skilled in the art should realize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, for the sake of clarity and conciseness, the description in the following description omits the description of well-known functions and structures. In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0046] The term "and / or," as used in this application, encompasses any and all combinations of one or more of the associated listed items. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. "Coupled" or "connected" or similar terms are not restricted to physical or mechanical connections or associations, but can also include electrical connections, whether direct or indirect.

[0047] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0048] Technical terms analysis:

[0049] 1. Equipment monitoring modality (E-Modality)

[0050] Refers to data derived from the monitored equipment or its attached sensors, used to characterize its operating state, health, or abnormal conditions, including but not limited to vibration waveform, temperature time series, pressure, current, voltage, etc. signal data.

[0051] 2. Personnel monitoring modality (P-Modality)

[0052] Refers to data used to monitor the position, posture, behavior, and physiological state of personnel in the target area, including but not limited to video image data, millimeter wave / laser radar point cloud data, ultra-wideband (UWB) positioning coordinates, wearable sensor data, etc.

[0053] 3. Environment monitoring modality (H-Modality)

[0054] Refers to data used to monitor the physical or chemical state of the environment in the target area, used to identify environmental risks that affect safety, including but not limited to temperature and humidity data, harmful gas concentration, dust concentration, illumination, noise intensity, etc.

[0055] 4. Modality set (M)

[0056] A set of three types of modalities (E, P, H) associated with the same monitoring object (e.g. a specific work station or area) pre-determined for a specific safety monitoring scenario (e.g. a construction site, a pipeline).

[0057] 5、Window-level fusion record )

[0058] A structured data record generated by the system at the end of each pre-defined time window, about the equipment-personnel-environment state of the monitoring object within this window. Each record contains feature fields corresponding to the three types of modalities and identifiers.

[0059] 6、Edge Inference Node

[0060] An AI computing device deployed at the edge of the production scene (e.g. an industrial computer, an intelligent gateway), responsible for receiving raw multi-modal data, performing real-time computation of lightweight inference models, and generating standardized modal feature data.

[0061] 7、Modal Feature i (t)

[0062] A high-level, structured feature representation obtained after pre-processing and AI inference of raw monitoring data. It contains timestamps, object identifiers, and model-inferred risk scores, state classifications, or abnormal feature vectors.

[0063] 8、Dynamic Risk Graph

[0064] A risk assessment structure constructed based on probabilistic graph models such as Bayesian networks, whose nodes correspond to risk factors such as equipment, personnel, and environment, and whose edges represent risk influence relationships, with parameters dynamically updated with real-time fusion data.

[0065] 9、Coupled Risk Identification

[0066] A complex, comprehensive safety risk triggered by a single modal anomaly, or by a specific logical combination of equipment, personnel, and environment anomalies.

[0067] Traditional safety monitoring methods can be mainly divided into the following categories:

[0068] 1、Single-point / single-modality sensing system: Install vibration, temperature sensors at key parts of equipment, or install video surveillance cameras in key areas. This type of system can only provide a single observation dimension and lacks risk correlation. For example, video surveillance can detect personnel violations, but cannot determine whether the equipment near them is in an abnormal operating state.

[0069] 2. Simple multi-system linkage: Establish a central monitoring platform to display the pictures or alarm information of the device monitoring system (SCADA), personnel positioning system and environmental monitoring system side by side. This kind of solution realizes "parallel monitoring", but does not realize "fusion analysis". The operator needs to judge the correlation between different subsystem alarms by himself, which is low in efficiency and easy to miss key clues.

[0070] 3. Centralized AI analysis based on cloud: Upload all sensor data (such as video stream) to cloud server and use deep learning model for unified analysis. Although this solution has certain intelligence, it faces serious real-time challenges. Network transmission delay and cloud processing queue delay make the period from event occurrence to alarm generation may be as long as several minutes, which cannot meet the immediate response needs of high-risk scenarios. At the same time, massive video data transmission occupies a large amount of network bandwidth.

[0071] 4. Fixed logic fusion based on rules: Try to fuse multi-modal data, but use simple "if-then" rules, such as "if device temperature > threshold, and there is someone nearby, then alarm". Such rule library lacks flexibility, is difficult to handle slightly complex coupling logic, has high maintenance cost, and lacks the ability to identify complex risk scenarios that are not predefined.

[0072] In summary, the existing technology in solving the coupled risk monitoring in complex scenarios faces the following core defects:

[0073] 1. Cross-modal data fusion level: Lack of a standard, structured method to organize device, personnel, and environment, three types of heterogeneous data with huge differences in time frequency and data format, making it difficult to ensure the generation of input structure that can be used for subsequent stable analysis of AI model.

[0074] 2. Intelligent analysis and real-time level: Cloud-based intelligent analysis cannot meet the high real-time requirements of high-risk scenarios, while simple edge computing is often limited by computing power, making it difficult to deploy complex AI models for real-time and accurate multi-modal analysis.

[0075] 3. Risk modeling level: Risk models are mostly static or based on single modal, unable to dynamically depict the complex influence relationship between device, personnel and environmental dynamic parameters, and lack of modeling, reasoning and prediction ability for coupled risks.

[0076] Therefore, the present application proposes a device-personnel-environment trinity safety monitoring scheme based on artificial intelligence, the core idea of which includes:

[0077] 1. Standardized window-level data fusion: Divide continuous data stream into equal time windows along time axis. At the end of each window, generate a structured fusion record about the monitoring object, which contains fixed feature fields of three modalities: equipment, personnel, and environment. Even if some modality data is temporarily missing in a window, reserve the position for that modality and mark the missing state, thus generating a series of risk analysis "snapshots" that are completely consistent in structure and continuous in time.

[0078] 2. Edge lightweight inference model deployment: Model compression (such as pruning, quantization) is performed on the three modality-specific AI models (equipment monitoring model, personnel behavior model, and environment risk model) trained and optimized on the cloud to convert them into lightweight versions and deploy them on edge AI intelligent devices close to the data source. The edge node is responsible for real-time processing of raw data streams to generate standardized modality features.

[0079] 3. Dynamic risk coupling analysis and early warning: In the edge or near-edge computing node, a risk model connecting the three types of modality features is constructed using dynamic Bayesian networks and other probabilistic graph models. This model dynamically updates the probability of each type of risk occurring and the influence strength between them based on the real-time influx of fusion records, enabling real-time inference and early warning of potential coupled risks.

[0080] Reference Figure 1 and Figure 2 One embodiment of the present application proposes a device-personnel-environment trinity safety monitoring method based on artificial intelligence, which can specifically include the following core steps.

[0081] S1, determine the modality set for a specific monitoring object; the modality set includes equipment monitoring modality, personnel monitoring modality, and environment monitoring modality.

[0082] Specifically, during system initialization, define the associated modality set M for each physical area or key equipment (i.e., "monitoring object") that needs to be monitored. For example, for a "chemical plant reaction kettle workstation", its modality set M is defined as:

[0083] Equipment monitoring modality (E): data collected or processed by reaction kettle body vibration sensors, bearing temperature sensors, etc.

[0084] Personnel monitoring modality (P): data collected or processed by intelligent cameras deployed above the workstation, UWB positioning tags worn by workers, etc.

[0085] Environment monitoring modality (H): data collected or processed by temperature and humidity sensors, hydrogen sulfide (H2S) gas concentration sensors, etc. deployed around the reaction kettle.

[0086] Then, the time window is opened and real-time collection is started. The system starts a preset length (e.g. 5 seconds) time window W1. During the window duration, all the aforementioned defined sensors work at their respective frequencies continuously, collecting raw data streams.

[0087] This step is completed by the system's modal set determination module, which defines the target and scope of all subsequent data collection, processing and analysis.

[0088] S2, through the inference node deployed on the edge side, real-time inference is performed on the raw data corresponding to the collected modal set to generate standardized modal features.

[0089] This step is the core function of the edge inference module. In this embodiment, the "cloud-edge collaboration" paradigm is adopted. In the cloud, three complex deep neural network models are respectively trained using massive historical data: one for device fault early warning (analyzing vibration spectrum), one for personnel behavior recognition (analyzing video), and one for environmental risk assessment (analyzing multi-parameter time series). Subsequently, through model pruning and quantization techniques, these three models are converted into lightweight versions. These lightweight models are deployed on the edge AI gateway (i.e. "edge inference node") close to the reaction kettle workstations.

[0090] Within the time window W1, the edge gateway runs these three lightweight models in parallel:

[0091] The device model receives the vibration waveform and outputs standardized features in real time, such as F E (t1): vibration level - low, F E (t2): vibration level - medium.

[0092] The personnel model receives the video stream and UWB coordinates and outputs features, such as F P (t1): {position: safety zone, behavior: normal patrol}, F P (t2): {position: operation zone, behavior: bending operation}.

[0093] The environment model receives sensor data and outputs features, such as F H (t1): {temperature: normal, H2S concentration: 5 ppm}, F H (t2): {temperature: normal, H2S concentration: 12 ppm}.

[0094] Each feature is accompanied by a high-precision timestamp. The key effect of this step is to convert massive, unstructured raw data (especially video streams) into a small amount of structured features rich in semantics locally, thereby greatly reducing the amount of data to be uploaded and controlling the feature extraction delay to the level of hundreds of milliseconds.

[0095] S3, in a preset time window, all modal features generated in the same time window and belonging to the same monitoring object are fused to generate a structured window-level fusion record; the window-level fusion record contains fixed fields for representing the device, personnel and environment states of the monitoring object in the time window.

[0096] The window end triggers the generation of a unified structure fusion record (R k ), which is the core function of the fusion record generation module. When the 5-second time window W1 ends, the system immediately triggers the fusion operation. The engine collects all F E , F P and F H features generated in the window. For each type of modal, a preset rule is applied to generate the representative value of the window. For example, the rule can be defined as "take the last valid feature value in the window". Assuming that at the last moment of the W1 window, the modal features are: F E (t end ): vibration level - medium, F P (t end ): {position: operating area, behavior: bending operation}, F H (t end ): {temperature: normal, H2S concentration: 12 ppm}. Then a fusion record R1 is generated:

[0097] R1 = {monitoring object ID: reactor-01, time window: W1, F E : vibration level - medium, F P : {position: operating area, behavior: bending operation}, F H : {temperature: normal, H2S concentration: 12 ppm}}

[0098] The application emphasizes the structural enforceability of the fusion record. Suppose in another window W2, the camera is disturbed by water mist and no valid personnel feature is output throughout the window, the fusion engine will still generate a record R2, and fill in a special "Missing" label in the F P field, thereby ensuring the continuity of the time series and the uniformity of the data structure. This mechanism is one of the cores of the application, which ensures that the downstream analysis engine always receives "data packets" with consistent structure and time alignment, thereby completely solving the structured input problem of multi-source heterogeneous data fusion.

[0099] S4, based on the window-level fusion record, performing risk coupling analysis through a dynamic risk probability graph model to output the coupling risk identification and risk probability of the monitoring object.

[0100] This step is the core of the risk analysis module. This embodiment adopts a pre-constructed Bayesian network as the dynamic risk probability graph model. The network contains evidence nodes (such as "equipment vibration", "distance between personnel and hazard source", "toxic gas concentration") and risk nodes (such as "mechanical injury risk", "poisoning risk"). The directed edges between nodes represent the causal relationship of risk influence, and are equipped with conditional probability tables (CPT) learned from historical accident data.

[0101] The fusion record R1 is input into the model. Specifically, the value of F E is mapped to the "equipment vibration" node state, the position information in F P is mapped to the "personnel distance" node state, and the concentration value in F H is mapped to the "toxic gas concentration" node state. Based on these "evidence", the model dynamically updates the posterior probability of nodes such as "poisoning risk" using the Bayesian inference formula. Assuming that the probability of "poisoning risk" is calculated to be 85%, which is much higher than the preset threshold of 60%.

[0102] The system then generates a coupled risk identification, such as "high poisoning risk exists in the reaction kettle-01 area", with a risk probability value. Further, the system can perform risk tracing and generate warning information based on probability contribution analysis: "main risk source: environmental H2S concentration seriously exceeds the standard (12 ppm); aggravating factors: personnel are operating near the leakage source". This warning information can immediately drive various safety responses: triggering sound and light alarms in the reaction kettle area, pushing detailed warning work orders to the team leader and safety officer mobile terminals, and automatically starting the emergency ventilation system of the work station.

[0103] This embodiment solves the data island problem through "window-level forced structured fusion", meets the real-time requirement through "edge lightweight inference", and realizes precise and quantitative identification and warning of coupled risks of equipment, personnel, and environment through "dynamic probability graph model".

[0104] In one embodiment of the construction site tower crane monitoring scene, the modal set M of a monitoring object contains: tower crane black box data (E), video monitoring below the hook (P), anemometer data (H). Within a certain 10-second time window, due to the rotation of the tower crane causing the video cable to be temporarily loose, the camera data is missing. When processing, the fusion record generation module takes the window mean value for the tower crane load and amplitude data (E), takes the maximum value for the wind speed (H), and for the personnel monitoring (P), since there is no valid input throughout the window, a structured tag is generated in the F P field: {state: "Data_Missing", confidence: 0}.

[0105] This record R missingStill be sent into the risk analysis model. The risk analysis model can be based on its built-in uncertainty processing mechanism, for example, when the personnel state evidence is missing, only according to the two pieces of evidence of equipment overload and excessive wind speed, the probability of "overturning risk" may be calculated as 50% (medium risk), instead of directly ignoring the window or causing calculation errors. This makes the system still maintain basic risk perception ability when part of the sensors are temporarily disabled, thereby significantly enhancing the robustness and continuity of the system in harsh industrial environments.

[0106] In some embodiments, in a large metallurgical plant area, the deployment architecture is divided into three layers:

[0107] 1. Edge layer: A large number of low-power AI cameras and sensor gateways are deployed near key equipment such as blast furnaces and rolling mills, running lightweight models to extract standardized features such as "molten steel splashing", "personnel not wearing protective clothing", and "high temperature in the area" in real time.

[0108] 2. Near-edge layer: A high-performance industrial control server (near-edge computing node) is deployed in each workshop in the plant area. All standardized features generated by edge devices in this workshop are aggregated to this server through industrial Ethernet.

[0109] 3. Cloud: Plant headquarters data center.

[0110] The near-edge server in the workshop undertakes the core task of fusion record generation and dynamic risk analysis. It generates a fusion record for the entire workshop every 3 seconds and runs a large Bayesian network model covering all risk patterns in the workshop (such as burns, mechanical injuries, and gas leaks). The model has high complexity and requires near-edge server-level computing power, but is deployed in the workshop, thereby avoiding network latency and bandwidth pressure caused by uploading all plant data to the cloud.

[0111] When the near-edge server analyzes that there is a high-risk combination of "molten iron splashing (E anomaly) + personnel staying (P anomaly)" in the tapping hole area of a blast furnace, it can control the warning light in that area to flash and broadcast evacuation instructions within 1 second. This architecture design sinks lightweight feature extraction to the most edge to ensure real-time performance and places complex coupling analysis in the near edge to ensure intelligent depth, which is an important innovation in the system design of the present application, effectively balancing computing load, real-time requirements, and analysis capabilities.

[0112] For the fire risk of underground comprehensive pipe gallery, one embodiment of the present application constructs a Bayesian network. The evidence nodes include: "cable temperature (E)", "oxygen concentration in the pipe gallery (H)", "whether there is personnel inspection (P)", "smoke concentration (H)". The risk node is "fire risk". Historical data shows that "high cable temperature" and "smoke concentration" have a direct impact on fire risk, while "personnel inspection" can usually reduce the risk (early detection), but when "oxygen concentration is high", it will exacerbate the spread of fire (combustion-supporting).

[0113] In the early stage of system operation, the model parameters (CPT) are initialized based on design experience and historical statistical data. After running for a month, the system accumulates hundreds of early warning records R k and the results of subsequent manual confirmation (true / false positives). These data are uploaded to the cloud to form a labeled risk sample database.

[0114] Every month, the Bayesian network is incrementally trained on the cloud using this database. For example, training finds that the combination of "moderate high cable temperature + no personnel inspection + normal oxygen concentration" does not cause a fire in practice many times, and the model will automatically adjust the conditional probability of the "fire risk" node under this combination. Conversely, if it is found that the combination of "local temperature slightly increased + abnormal concentration of a specific volatile organic compound (VOC)" which has not been paid attention to before is a precursor to a small-scale smoldering, the model will learn this new association and increase its risk probability.

[0115] The optimized model parameters are updated to all near-edge analysis nodes in synchronization. When the next time the system monitors F E : cable joint temperature rise - medium, F H : {VOC: slightly over standard, O2: normal}, F P : no personnel inspection, the updated model can more accurately calculate a higher fire risk probability and generate a traceable early warning: "Early warning: early risk of electrical fire. Main basis: abnormal cable joint temperature; Associated environmental evidence: abnormal VOC concentration, suggesting possible overheating decomposition of insulating materials." The risk model of the present embodiment not only can perform dynamic reasoning and cause tracing, but also can continuously self-learn and self-optimize through actual operation data closed-loop iteration, thereby continuously improving the accuracy of early warning and adaptability to new risk patterns.

[0116] In some embodiments, in the AGV (automatic guided vehicle) operation area of the automated warehouse, the original data of the personnel monitoring mode (P) is high-definition video. The lightweight model running on the edge reasoning node is a lightweight human pose estimation algorithm (such as MobilePose), and the standardized modal feature F P(t) is not simply "someone / no one", but a time series of two-dimensional coordinates of human body skeleton key points (such as 17 points of head, shoulder, elbow, wrist, etc.).

[0117] At the same time, the equipment monitoring mode (E) provides the real-time position, speed and planned path of the AGV. A special sub-model in the risk analysis module is responsible for real-time spatio-temporal correlation analysis of F P (human motion trajectory) and F E (AGV motion envelope). It calculates the spatial intersection probability of the predicted position of the human body skeleton points and the AGV motion path within the next 2 seconds.

[0118] When the analysis finds that although a staff member has not entered the current stationary area of the AGV, his walking trajectory and posture (such as looking down at the mobile phone) indicate that he will enter the path of an AGV that is accelerating in 1.5 seconds, the dynamic risk probability graph model will comprehensively consider the F P (predicted trajectory danger) and F E (AGV speed high) features at this moment, quickly calculate that the "person-vehicle collision risk" probability has sharply increased, and thus issue an early sound and light warning, or even send a deceleration instruction to the AGV. This embodiment shows how the application deeply integrates deep behavior understanding (posture estimation and trajectory prediction) with equipment state, thereby achieving early and accurate warning of dynamic interaction risks, which is a major progress that traditional area intrusion detection technology cannot achieve.

[0119] Overall, the beneficial effects of the present application compared to the prior art include:

[0120] 1. It realizes a fundamental change from "data island" to "structured fusion", laying a foundation for complex risk analysis.

[0121] Although the prior art conducts multi-source data collection and preliminary fusion, it lacks mandatory constraints on data structure. Through the core mechanism of "generating structured fusion records containing fixed fields of equipment, personnel and environment periodically with a fixed time window", the present application uniformly converts raw data streams of different sources, frequencies and formats into a series of analysis units with time series alignment and consistent fields. This solves the problem of multi-modal data fragmentation from the bottom, provides a reliable guarantee for subsequent advanced AI models (such as probability graph models) that rely on stable data input, and is a prerequisite for precise coupling risk analysis.

[0122] 2. From "single-point threshold warning" to "coupling probability warning", significantly improving the accuracy and depth of risk identification.

[0123] Traditional methods (such as rule-based fixed logic fusion) or part of existing technologies (such as using rule engine) mainly rely on single-dimensional threshold or simple logic combination, with high false positive rate and unable to quantify complex risks. The application uses dynamic probabilistic graph models (such as Bayesian networks) to analyze structured fusion records, which can explicitly model and calculate the complex causal and statistical relationships between equipment, personnel, and environmental states. This enables the system not only to identify simple combinations of "A and B", but also to quantitatively assess complex coupling scenarios such as "when A occurs, how much does B aggravate the risk of C", thus realizing the leap from qualitative judgment to quantitative assessment of risk, significantly reducing the false positive rate and discovering previously overlooked implicit associated risks.

[0124] 3. Through the "cloud-edge collaborative lightweight inference" architecture, the dual requirements of "high real-time" and "high intelligence" in high-risk scenarios are met.

[0125] Some existing technologies do not explicitly deploy edge lightweight, while pure cloud processing has the problem of high delay. The application innovatively adopts the collaborative paradigm of "cloud training optimization and edge lightweight deployment". The computationally intensive model training is placed in the cloud to ensure model accuracy; the optimized lightweight model is sunk to the edge node to realize local real-time feature extraction of raw data. This avoids the long-distance transmission of massive video data, reduces the delay from "event occurrence" to "feature readiness" from minutes to milliseconds / seconds, enabling immediate response to instantaneous risks such as mechanical collision and personnel intrusion, and solving the real-time deficiency in high-risk industrial scenarios.

[0126] 4. The system is given the ability to "evolve continuously", from a "static system" to a "self-adaptive system".

[0127] Most existing monitoring systems have fixed rules or models after deployment, making it difficult to adapt to new working conditions. The application collects historical warning data and feedback, builds a risk sample database, and periodically performs incremental training and parameter optimization on the dynamic risk probability graph model. This enables the system to learn new risk patterns from actual operation, automatically adjust risk judgment logic, and continuously improve warning accuracy in long-term operation, adapt to changes in production processes and new risks brought by new equipment commissioning, thus realizing self-learning and self-adaptation of safety monitoring systems.

[0128] 5. The system's "robustness" in abnormal working conditions is enhanced, ensuring the continuity of monitoring.

[0129] In real-world industrial environments, temporary sensor failures and network outages are frequent occurrences. This application employs a "missing state marking" mechanism for missing data when generating fusion records, rather than directly discarding the time window or the entire data stream. This ensures the continuity of the analyzed time series, enabling the risk analysis model to perform uncertainty reasoning based on existing information even with partially missing evidence, and still output valuable risk assessment results. This significantly improves the reliability and availability of the system in complex and harsh field environments.

[0130] These beneficial effects are interconnected and progressive, together constituting the outstanding technological progress and practical value of the technical solution of this application compared with the prior art.

[0131] The embodiments of the safety monitoring methods and the embodiments of the safety monitoring systems are identical or related in technical concept. They can be referenced and learned from each other in terms of technical details and technical effects. The embodiments of the safety monitoring systems have actually been incorporated into the embodiments of the safety monitoring methods, so they will not be described again.

[0132] Based on the same inventive concept, embodiments of this application also provide an electronic device. Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. Figure 3 As shown in the embodiments of this application, an electronic device includes: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the security monitoring methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0133] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0134] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0135] In some embodiments, the one or more processors 101 comprise a field programmable gate array.

[0136] Embodiments of the present application also provide a computer readable medium. The computer readable medium stores a computer program, wherein the program is executed by a processor to implement the steps in any of the above-mentioned safety monitoring methods. The computer readable storage medium can be a volatile or non-volatile computer readable storage medium.

[0137] Embodiments of the present application also provide a computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-mentioned safety monitoring method.

[0138] Those of ordinary skill in the art will understand that all or some of the steps in the above-disclosed methods, the functions of the various systems, devices, and modules described above can be implemented as software, firmware, hardware, or any suitable combination thereof. In a hardware implementation, the division of functionality between the functional modules / components referred to in the above description does not necessarily correspond to physical divisions of hardware components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components working together. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer readable storage media, which can include computer storage media (or non-transitory media) and communication media (or transitory media).

[0139] As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read only memory (CD-ROM), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Further, it should be understood by those of ordinary skill in the art that communication media typically embodies computer readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. In the above description of various example embodiments, numerous specific details are set forth to provide a thorough understanding of various example implementations. However, it will be apparent to those skilled in the art that not all of the specific details are necessarily required in every implementation, and that certain process steps can not be required. Further, those of ordinary skill in the art will recognize that the processes described above can be embodied in a variety of forms, including but not limited to routine, subroutine, thread of execution, process, program, or other software components. It will also be recognized that a "process" or "routine" can be implemented as a hardware routine, as a software routine running on a processor, or as a combination of hardware and software routines. In addition, although the various functional modules, components, and units described above can be implemented as software, hardware, or a combination thereof, the various functional modules, components, and units are shown as software modules in the drawings for ease of illustration and understanding.

[0140] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0141] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0142] The computer program product described herein can be embodied in a specific manner by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied specifically as a computer storage medium, and in another optional embodiment, the computer program product is embodied specifically as a software product, such as a software development kit (SDK) or the like.

[0143] The computer program product described herein can be embodied in a specific manner by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied specifically as a computer storage medium, and in another optional embodiment, the computer program product is embodied specifically as a software product, such as a software development kit (SDK) or the like.

[0144] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium can also have instructions stored thereon or therein which may

[0145] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0146] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0147] Exemplary embodiments have been disclosed and, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation, unless otherwise specifically stated. In some embodiments, it will be apparent to one of ordinary skill in the art that features, characteristics, and / or elements described in conjunction with a particular embodiment can be used in conjunction with other embodiments unless otherwise specifically noted. As such, one of ordinary skill in the art will understand that various changes in form and details can be made without departing from the scope of the present application as set forth in the appended claims.

Claims

1. A safety monitoring method, characterized by, The method comprises the following steps: determining a modal set for a specific monitoring object; the modal set comprises a device monitoring modal, a personnel monitoring modal and an environment monitoring modal; generating standardized modal features by real-time inference on raw data corresponding to the modal set collected by an inference node deployed on an edge side; fusing all modal features generated within a same time window and belonging to a same monitoring object to generate a structured window-level fusion record; the window-level fusion record comprises fixed fields for representing device, personnel and environment states of the monitoring object within the time window; based on the window-level fusion record, performing risk coupling analysis by a dynamic risk probability graph model to output a coupling risk identification and a risk probability of the monitoring object.

2. The safety monitoring method of claim 1, wherein, The step of generating a structured window-level fusion record comprises: for each type of modal in the modal set, if there is a corresponding valid modal feature in the current time window, a representative feature value of the type of modal in the current window is extracted or calculated from the valid modal feature based on a preset rule; if there is no valid modal feature of a type of modal in the current time window, a missing state marker is generated for the type of modal; the representative feature value or the missing state marker is filled into the corresponding fixed field of the window-level fusion record in a preset order.

3. The safety monitoring method according to claim 1 or 2, characterized in that, The step of generating standardized modal features by the inference node deployed on the edge side comprises: training special artificial intelligence models corresponding to device, personnel and environment modals based on historical data in the cloud; performing lightweight processing on the trained special artificial intelligence models to obtain lightweight inference models; deploying the lightweight inference models to the inference node on the edge side, loading and running the inference node to infer real-time raw data flowing in, and outputting the standardized modal features.

4. The safety monitoring method of claim 1, wherein, The dynamic risk probability graph model is a Bayesian network model, and its nodes include evidence nodes corresponding to device states, personnel behaviors and environment parameters, and risk nodes representing different coupling risk types; The step of performing risk coupling analysis comprises: inputting the modal feature values in the window-level fusion record into the corresponding evidence nodes of the Bayesian network model; dynamically updating the posterior probabilities of the risk nodes according to the evidence and the preset node conditional probability table in the model; determining the risk represented by the risk node whose posterior probability exceeds a preset threshold as the coupling risk identification, and outputting its posterior probability as the risk probability.

5. The safety monitoring method of claim 4, wherein, Further comprising: when the risk probability exceeds a warning threshold, generating warning information containing risk cause tracing information according to the risk node type triggering the warning and the corresponding evidence node state; the warning information is used to drive a corresponding safety response operation.

6. The safety monitoring method of claim 1, wherein, Further comprising: collecting historical warning events and their corresponding window-level fusion records to build a labeled risk sample database; using the risk sample database to periodically incrementally train and optimize parameters of the dynamic risk probability graph model.

7. The safety monitoring method of claim 1, wherein, The standardized modality feature of the personnel monitoring modality is a human skeleton key point time sequence trajectory feature generated by a pose estimation algorithm based on video image data; The dynamic risk probability graph model analyzes the human skeleton key point time sequence trajectory feature and the equipment operation feature in time and space to identify the collision risk of personnel entering the equipment motion envelope.

8. A safety monitoring system capable of implementing the safety monitoring method according to any one of claims 1 to 7, characterized in that, Comprise: A modality set determination module is configured to determine a modality set for a specific monitoring object; the modality set includes an equipment monitoring modality, a personnel monitoring modality, and an environment monitoring modality; An edge inference module is deployed on the edge side and is configured to perform real-time inference on raw data corresponding to the modality set collected to generate standardized modality features; A fusion record generation module is configured to fuse all modality features generated within a same time window and belonging to a same monitoring object in a preset time window to generate a structured window-level fusion record; the window-level fusion record includes fixed fields for representing equipment, personnel, and environment states of the monitoring object within the time window; A risk analysis module is configured to perform risk coupling analysis on the window-level fusion record based on a dynamic risk probability graph model to output a coupling risk identification and a risk probability of the monitoring object.

9. An electronic device, comprising: Comprise: One or more processors; Memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors can implement the steps in the safety monitoring method of any one of claims 1 to 7.

10. A computer readable medium having stored thereon a computer program, characterized in that The computer program can implement the steps in the safety monitoring method of any one of claims 1 to 7 when executed by a processor.

Images