Laboratory safety monitoring method and system based on multi-sensor data

Abstract

The present application relates to the technical field of data processing, in particular to a laboratory safety monitoring method and system based on multi-sensor data. The method comprises: obtaining standardized data of dimensional data of multi-source sensors deployed at different positions in the laboratory at multiple sampling time points; obtaining a fusion data stream at each sampling time point; constructing a dynamic heterogeneous graph in the laboratory; using the fusion data stream to update the node state of the dynamic heterogeneous graph in real time, and using a graph neural network model to process the dynamic heterogeneous graph to extract a graph feature vector sequence representing the overall state of the laboratory scene; inputting the graph feature vector sequence into a time series prediction model to output a risk vector for monitoring the safety risk of the laboratory environment. That is, the scheme of the present application can safely and accurately monitor the safety of the laboratory environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing. More particularly, the present application relates to a laboratory safety monitoring method and system based on multi-sensor data. BACKGROUND

[0002] Laboratories, as important places for scientific research and technological development, usually integrate precise instruments, chemical reagents, high-temperature and high-pressure equipment, and complex pipeline systems, making them a highly concentrated environment of potential risks.

[0003] Traditional laboratory safety management relies on rules and regulations, safety education, and manual regular inspections. This mode is highly subjective and is easily affected by personnel's responsibility, experience, and fatigue state, making it difficult to achieve real-time monitoring around the clock and in all corners. The response to sudden and hidden safety incidents is slow. In order to make up for the shortcomings of manual management, some laboratories have introduced independent sensor alarm systems, such as smoke detectors, temperature and humidity sensors, and combustible gas leakage alarms. However, these sensors usually exist as isolated "information islands" and are self-governed, only able to issue threshold alarms for single physical or chemical quantity exceeding events, and are unable to comprehensively analyze multiple factors. For example, a temperature sensor cannot distinguish between normal operation heat dissipation and initial fire, resulting in a high false alarm rate, and is unable to effectively identify complex risk scenarios caused by the coupling of multiple factors (such as personnel misoperation near dangerous chemical cabinets).

[0004] With the development of Internet of Things, computer vision, and artificial intelligence technologies, modern laboratory safety monitoring has begun to evolve towards intelligence. For example, video monitoring systems combined with intelligent analysis algorithms can identify human posture, abnormal intrusion, or left objects; and Internet of Things sensor networks can remotely monitor environmental parameters. Although these technologies have improved monitoring efficiency to some extent, existing solutions still have limitations, as follows:

[0005] On the one hand, different types of data (such as video images, environmental parameters, and personnel positioning beacons) come from various sources and have non-uniform time and space references. How to effectively fuse these heterogeneous and asynchronous data to form a unified and coherent understanding of the scene is a technical challenge.

[0006] On the other hand, existing systems often lack the ability to model the complex relationships between the "scene-objects-people" in the laboratory, and are unable to associate isolated data points with deep semantic information such as laboratory physical layout, equipment function, and personnel authority. Therefore, it is difficult to understand the context of events and can only make superficial judgments. More importantly, most systems still focus on post-event or immediate alarms and lack the ability to predict the evolution trend of risks. They are unable to analyze a series of weak abnormal signs and issue early warnings before safety incidents occur.

[0007] Therefore, there is an urgent need for a new monitoring method that can deeply integrate multi-source heterogeneous data, understand the dynamic relationship between entities in the scene, and have the ability of forward-looking risk prediction, so as to realize comprehensive, accurate and proactive control of the laboratory safety state. SUMMARY

[0008] The purpose of the present application is to propose a laboratory safety monitoring method and system based on multi-sensor data, in order to solve the problem that the prior art cannot comprehensively and accurately monitor laboratory safety; for this purpose, the present application provides a scheme in the following two aspects.

[0009] In the first aspect, the present application provides a laboratory safety monitoring method based on multi-sensor data, comprising:

[0010] Obtaining standardized data of dimensional data of multi-source sensors deployed at different positions in the laboratory at multiple sampling time points; fusing the standardized data of multiple dimensions at each sampling time point through a spatio-temporal fusion model to obtain a fusion data stream at each sampling time point;

[0011] Constructing a dynamic heterogeneous graph in the laboratory, wherein the nodes in the graph include devices, personnel and environmental areas, and the connection edges between the nodes in the graph represent the physical adjacency or functional interaction relationship between the nodes;

[0012] Using the fusion data stream to update the node state of the dynamic heterogeneous graph in real time, and using a graph neural network model to process the dynamic heterogeneous graph to extract a graph feature vector sequence representing the overall state of the laboratory scene;

[0013] Inputting the graph feature vector sequence into a time series prediction model to output a risk vector; the risk vector is a risk probability of multiple risk types; when the risk probability of any risk type exceeds the corresponding preset safety threshold, it is determined that the laboratory has a safety risk of the corresponding risk type.

[0014] Optionally, the construction of the dynamic heterogeneous graph in the laboratory comprises: dividing the laboratory space into grid units of a preset size as nodes corresponding to the environmental areas, and creating nodes corresponding to the devices and nodes corresponding to the personnel according to the device information and the positioning information of the personnel; when the spatial distance between any two nodes is less than a first preset threshold, a physical adjacency edge is established; when the distance between the node corresponding to the personnel and the node corresponding to the device is less than a second preset threshold, and the interaction time of the two nodes exceeds an interaction time threshold, a functional interaction edge is established; when the personnel are located in the environmental area, a subordinate relationship edge is established between the node corresponding to the personnel and the node corresponding to the environmental area; the first preset threshold is greater than the second preset threshold.

[0015] Optionally, the updating the dynamic heterogeneous graph by using the fused data stream comprises: updating the environment parameters and the personnel behavior states contained in the fused data stream into attributes of nodes corresponding to the environment regions and nodes corresponding to the personnel in the graph respectively, to obtain an updated dynamic heterogeneous graph.

[0016] Optionally, the time series prediction model is a long short-term memory network.

[0017] Optionally, the fusing the standardized data of multiple dimensions at each sampling time by using the spatio-temporal fusion model comprises:

[0018] The fused data stream of optimal estimation is output after the data fusion of the standardized data of multiple dimensions at each sampling time by using the Kalman filtering algorithm.

[0019] Optionally, the risk types at least include a fire risk, a chemical leakage risk or a personnel operation risk.

[0020] Optionally, the standardized data is obtained by using a normalization method to standardize the data of each dimension; and the normalization method is a maximum-minimum value normalization method or a Z-score normalization method.

[0021] Optionally, the method further comprises the step of aligning the standardized data of multiple dimensions at each sampling time by using the spatio-temporal fusion model, specifically: aligning time stamps of the standardized data of multiple dimensions at each sampling time to a preset reference sampling frequency, and using a predetermined interpolation method to fill in data of low-frequency dimensions, to realize time synchronization.

[0022] Optionally, the multi-source sensors comprise a temperature and humidity sensor, a gas leakage detector, a monitoring camera and an ultra-wideband (UWB) beacon worn by personnel.

[0023] In a second aspect, a laboratory safety monitoring system based on multi-sensor data comprises:

[0024] a processor;

[0025] a memory storing computer instructions for laboratory safety monitoring based on multi-sensor data, when the computer instructions are run by the processor, the system executes the laboratory safety monitoring method based on multi-sensor data described above.

[0026] The present application has the following advantages:

[0027] The application aligns and integrates the environment, video and personnel position data of different sources and different time sequences in the laboratory through a space-time fusion model, overcomes the disadvantages of data islands and one-sided information in the traditional monitoring scheme, and provides a unified and high-quality data basis for subsequent comprehensive analysis. Further, a heterogeneous graph representing the laboratory environment is constructed, and a graph neural network model is used for processing, so that the physical association and functional interaction between entities can be deeply understood, and the potential risk signs caused by multi-factor coupling can be more accurately captured. Then, a time series prediction model is introduced to analyze the evolution law of the overall state characteristics of the laboratory, realize the forward prediction of future risks, and improve the safety management from passive immediate alarm to proactive early warning. The specific risk type can be warned in advance to achieve the purpose of prevention, greatly enhancing the initiative and effectiveness of laboratory safety management. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A step flowchart of the laboratory safety monitoring method based on multi-sensor data in the embodiment is schematically shown;

[0029] Figure 2 A structural block diagram of the laboratory safety monitoring system based on multi-sensor data in the embodiment is schematically shown. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application.

[0031] As shown in the figure, Figure 1 The laboratory safety monitoring method based on multi-sensor data in the embodiment includes the following steps:

[0032] Step S1, obtaining standardized data of dimensional data of multi-source sensors deployed at different positions in the laboratory at multiple sampling time points; fusing the standardized data of multiple dimensions at each sampling time point through a space-time fusion model to obtain a fusion data stream at each sampling time point.

[0033] Specifically, the multi-source sensors include temperature and humidity sensors, gas leakage detectors, monitoring cameras and ultra-wideband (UWB) beacons worn by personnel.

[0034] Among them, the temperature and humidity sensors and the gas leakage detectors deployed in the laboratory are used to obtain environmental data (temperature, humidity, gas concentration), the monitoring cameras are used to obtain video stream data, and the ultra-wideband (UWB) beacons worn by personnel are used to obtain personnel position data.

[0035] The network time protocol (NTP) can be used to synchronize the timestamps of all sensors. Timestamp synchronization aims to unify different frequency sensor data to a standard time axis. For example, a high-speed camera has a sampling frequency of 30 frames per second, and a temperature sensor has a frequency of 1 Hz, i.e., 1 time per second. Therefore, timestamp synchronization is needed.

[0036] The spatio-temporal fusion model is a Kalman filter algorithm. Kalman filter is a linear minimum variance estimator used to estimate random processes or systems in the presence of noise. Its main principle is to continuously optimize the estimation of the system state through a series of iterative steps. The main steps of the algorithm are as follows:

[0037] Prediction: Estimate the next state based on the current system state estimate and noise.

[0038] Update: Update the system state estimate based on the new measurement and predicted state.

[0039] The Kalman filter algorithm can be applied to sensor data fusion and target tracking.

[0040] In this embodiment, the process of fusing the standardized data of multiple dimensions at each sampling time through the spatio-temporal fusion model is as follows:

[0041] Based on the video data frame rate, the low-frequency sampled environmental data and position data are interpolated, and the multi-source data in the adjacent area is weighted and averaged according to the position of each sensor in the pre-constructed laboratory three-dimensional coordinate system, generating a unified feature vector synchronized with the video frame and containing the state information of all entities in the scene as the fusion data stream. The fusion data stream includes the optimal estimated environmental data (temperature and harmful gas concentration of each environmental area) and position data (position of personnel).

[0042] The above weighted fusion determines the importance of the data source in the final analysis according to its reliability and correlation.

[0043] The standardized data is obtained by standardizing each dimension of data using a normalization method; the normalization method is the maximum and minimum value normalization method, the Z-score normalization method.

[0044] Step S2, a dynamic heterogeneous graph in the laboratory is constructed, the nodes in the graph include devices, personnel and environmental areas, and the connection edges between the nodes in the graph represent the physical adjacency or functional interaction relationship between the nodes.

[0045] Specifically, the nodes corresponding to the equipment are centrifuges, fume hoods, and gas cylinder cabinets, and the attributes thereof include switch states, powers, and operating temperatures; the nodes corresponding to the personnel are personnel in the indoor environment, and the attributes thereof include real-time positions and operation permissions; the nodes corresponding to the environmental areas are chemical storage areas and high-temperature experimental areas, and the attributes thereof include average temperatures and harmful gas concentrations. The types of the connection edges include physical adjacency edges, functional interaction edges, and subordinate relationship edges; the physical adjacency edge is, for example, a node corresponding to personnel and a node corresponding to equipment are less than a first preset threshold in distance; the functional interaction edge is, for example, a node corresponding to personnel and a node corresponding to equipment are less than a second preset threshold in distance, and the interaction duration between the two nodes exceeds an interaction duration threshold; and the subordinate relationship edge is, for example, a node corresponding to personnel is subordinate to a node corresponding to an experimental area.

[0046] The establishment of the edges in the figure defines the relationship between the nodes. The first preset threshold can be set to 2 meters, when the node P corresponding to the personnel walks to a distance of 1.5 meters from the node D1 (fume hood) corresponding to the equipment, the system will automatically establish a physical adjacency edge between the node P and the node D1, indicating that the two are close in space.

[0047] The second preset threshold can be set to 1 meter, and the interaction duration threshold can be set to 15 seconds. If the node P corresponding to the personnel not only approaches the node D1 (fume hood) corresponding to the equipment, but also stays within 1 meter of the node D1 (fume hood) and operates for more than 15 seconds, this behavior is recognized by the video analysis system, and the system will additionally establish a functional interaction edge between the two. This edge indicates that the two not only approach each other, but also interact, which is crucial for subsequent judgment of whether there is a risk of violating the operation.

[0048] In step S3, the node states of the dynamic heterogeneous graph are updated in real time using the fusion data stream, and a graph neural network model is used to process the dynamic heterogeneous graph to extract a graph feature vector sequence representing the overall state of the laboratory scene.

[0049] The updating of the dynamic heterogeneous graph using the fusion data stream includes updating the environmental parameters and the personnel behavior states included in the fusion data stream into the attributes of the nodes corresponding to the environmental areas and the nodes corresponding to the personnel in the graph, respectively.

[0050] For example, the updates to the dynamic heterogeneous graph reflect changes in laboratory conditions in real time. Suppose the fused data stream shows that the volatile organic compound concentration in the area corresponding to node E25 in the environmental region increases from 10 ppm to 50 ppm, and the temperature increases from 22 degrees Celsius to 28 degrees Celsius; then the attribute feature vector of node E25 will be updated with these new values. Simultaneously, if the fused data stream indicates that the behavior state of node P corresponding to a person changes from stationary to rapid movement, its corresponding node attributes will also change accordingly. This update is continuous, ensuring that the dynamic heterogeneous graph processed by the subsequent graph neural network model always reflects the latest and most accurate on-site situation.

[0051] In this embodiment, the updated dynamic heterogeneous graph is input into a graph neural network model, and the corresponding graph feature vector is output by aggregating the feature information of the nodes themselves and their neighbors. The graph neural network model can be a graph convolutional network model or a graph attention network model.

[0052] The graph neural network model in this embodiment performs in-depth analysis on the updated dynamic heterogeneous graph.

[0053] Taking a graph attention network as an example, when processing node P corresponding to a person, it considers not only node P's own rapid movement state but also its connected neighboring nodes. For instance, node P is connected to node E25 corresponding to the environment and node D corresponding to the equipment (gas cylinder cabinet). In this case, the graph attention network calculates the importance of neighboring nodes and finds that node E25 has an abnormal volatile organic compound concentration of 50 ppm, thus giving node E25 a higher attention weight. By aggregating node P's own state information and the weighted information of its important neighbors, the graph neural network model can generate a comprehensive graph feature vector.

[0054] In this embodiment, graph feature vectors at consecutive time points are collected, for example, one graph feature vector is generated per second, forming a graph feature vector sequence.

[0055] The graph feature vector sequence is constructed to capture the dynamic evolution of risks (e.g., rapid personnel movement is likely related to gas leaks). A graph feature vector is generated and collected every second. For prediction, 30 graph feature vectors collected over the past 30 seconds are taken and arranged chronologically to form a time-step sequence of length 30, denoted as the graph feature vector sequence. This time-step sequence records the evolution of the overall laboratory status over the past half-minute; for example, it might first detect the abnormal opening of a chemical cabinet door, followed by gradual changes in nearby air parameters, and finally, the abnormal gathering of personnel.

[0056] Step S4: Input the graph feature vector sequence into the time series prediction model and output the risk vector; the risk vector is the risk probability of multiple risk types; when the risk probability of any risk type exceeds the corresponding preset safety threshold, it is determined that the laboratory has a safety risk of the corresponding risk type.

[0057] In this embodiment, the graph feature vector sequence is input into a time-series prediction model (such as an LSTM model), and a risk vector is output. This risk vector is a multi-dimensional vector, with each dimension representing a type of risk. For example, a three-dimensional vector where the first dimension represents fire risk, the second dimension represents chemical spill risk, and the third dimension represents violation operation risk, with each dimension ranging from 0 to 1.

[0058] Among them, the time-series prediction model can effectively capture the long-term dependency of graph feature vectors over time (such as capturing the pattern of personnel leaving after equipment temperature rises slowly and continuously, indicating potential fire risk); and by analyzing the dynamic patterns of the entire sequence and based on the development trend over a period of time, it can predict the most likely risk situation at the next moment and output a multi-dimensional risk vector that includes assessments of multiple possibilities such as fire and leakage.

[0059] In one embodiment, based on laboratory safety management regulations and historical risk data, the risk types in this embodiment include fire risk, chemical spill risk, and personnel violation risk.

[0060] In this embodiment, the risk probability of each risk type in the risk vector output by the time series prediction model is compared with the corresponding preset safety threshold in real time.

[0061] Each risk type corresponds to a preset safety threshold. For example, the preset safety threshold for fire risk is 0.9, the preset safety threshold for chemical spill risk is 0.85, and the preset safety threshold for personnel violation risk is 0.6.

[0062] For example, if the risk vectors output by the time series prediction model are 0.92, 0.7, and 0.6, then because the first dimension of 0.92 exceeds the preset fire risk threshold of 0.9, the system determines that there is a fire risk in the laboratory and a high-priority fire warning needs to be triggered immediately.

[0063] The solution of this invention can issue early warnings for specific risk types, achieving the purpose of prevention and greatly enhancing the initiative and effectiveness of laboratory safety management, that is, enabling comprehensive and accurate monitoring of laboratory safety.

[0064] This invention also provides a laboratory safety monitoring system based on multi-sensor data. For example... Figure 2As shown, the system includes a processor and a memory, the memory storing computer program instructions, which, when executed by the processor, implement the laboratory safety monitoring method based on multi-sensor data according to the present invention.

[0065] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and therefore will not be described in detail here.

[0066] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented by computer-readable / executable instructions stored or otherwise maintained on such a computer-readable medium.

[0067] In the description of this specification, "multiple" means at least two, such as two, three or more, etc., unless otherwise expressly and specifically defined.

[0068] While various embodiments of the invention have been shown and described in this specification, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention.

Claims

1. A method for laboratory safety monitoring based on multi-sensor data, characterized in that, The method comprises the following steps: obtaining standardized data of multi-dimensional data of multi-source sensors deployed at different positions in the laboratory at multiple sampling time points; fusing the standardized data of multiple dimensions at each sampling time point through a spatio-temporal fusion model to obtain a fusion data stream at each sampling time point; constructing a dynamic heterogeneous graph in the laboratory, wherein the nodes in the graph include equipment, personnel and environmental areas, and the connection edges between the nodes in the graph represent the physical adjacency or functional interaction relationship between the nodes; updating the node state of the dynamic heterogeneous graph in real time using the fusion data stream, and processing the dynamic heterogeneous graph using a graph neural network model to extract a graph feature vector sequence representing the overall state of the laboratory scene; inputting the graph feature vector sequence into a time series prediction model to output a risk vector; the risk vector is a risk probability of multiple risk types; when the risk probability of any risk type exceeds the corresponding preset safety threshold, it is determined that the laboratory has a safety risk of the corresponding risk type; constructing a dynamic heterogeneous graph in the laboratory, including: dividing the laboratory space into grid units of a preset size as nodes corresponding to environmental areas, and creating nodes corresponding to equipment and nodes corresponding to personnel according to equipment information and personnel positioning information; when the spatial distance between any two nodes is less than a first preset threshold, a physical adjacency edge is established; when the distance between the node corresponding to the personnel and the node corresponding to the equipment is less than a second preset threshold, and the interaction time length of the two nodes exceeds an interaction time length threshold, a functional interaction edge is established; when the personnel are located in the environmental area, a subordinate relationship edge is established between the node corresponding to the personnel and the node corresponding to the environmental area.

2. The multi-sensor data based laboratory safety monitoring method of claim 1, wherein, The method for updating the dynamic heterogeneous graph using the fusion data stream comprises: updating the environmental parameters and personnel behavior states contained in the fusion data stream into the attributes of the nodes corresponding to the environmental areas and the nodes corresponding to the personnel in the graph, respectively, to obtain an updated dynamic heterogeneous graph.

3. The multi-sensor data based laboratory safety monitoring method of claim 1, wherein, The time series prediction model is a long short-term memory network.

4. The multi-sensor data based laboratory safety monitoring method of claim 1, wherein, The method for fusing the standardized data of multiple dimensions at each sampling time point through a spatio-temporal fusion model to obtain a fusion data stream at each sampling time point comprises: outputting an optimally estimated fusion data stream after performing data fusion of the multi-source sensors on the standardized data of multiple dimensions at each sampling time point through a Kalman filtering algorithm.

5. The multi-sensor data based laboratory safety monitoring method of claim 1, wherein, The risk types at least include fire risk, chemical leakage risk or personnel violation operation risk.

6. The multi-sensor data based laboratory safety monitoring method of claim 1, wherein, The standardized data is obtained by standardizing each dimension data using a normalization method; wherein the normalization method is a maximum and minimum value normalization method or a Z-score normalization method.

7. The multi-sensor data based laboratory safety monitoring method of claim 1, wherein, The method further comprises a step of aligning the standardized data of multiple dimensions at each sampling time point using the spatio-temporal fusion model, specifically: aligning the timestamps of the standardized data of multiple dimensions at each sampling time point to a preset reference sampling frequency, and using a predetermined interpolation method to fill in the data of low-frequency dimensions to realize time synchronization.

8. The multi-sensor data based laboratory safety monitoring method of claim 4, wherein, The multi-source sensors include temperature and humidity sensors, gas leakage detectors, monitoring cameras and ultra-wideband (UWB) beacons worn by personnel.

9. Laboratory safety monitoring system based on multi-sensor data, characterized in that, The method comprises the following steps: a processor; a memory storing computer instructions for laboratory safety monitoring based on multi-sensor data, which, when executed by the processor, cause the system to perform the method for laboratory safety monitoring based on multi-sensor data according to any one of claims 1-8.

Images