Intelligent laboratory safety monitoring method and system based on AIoT

Abstract

The invention discloses an AIoT-based intelligent laboratory safety monitoring method and system. The method comprises the steps of S1, collecting laboratory multi-source heterogeneous data and performing preprocessing; s2, respectively extracting gas diffusion characteristics and thermal infrared characteristics according to the preprocessed gas concentration sequence and thermal infrared image sequence, and then calculating fusion characteristics; s3, physical constraint characteristics are calculated according to gas diffusion characteristics and thermal infrared characteristics, and a risk thermodynamic diagram is generated; s4, extracting a personnel bounding box and personnel behavior characteristics according to the preprocessed personnel video sequence, calculating global risk coordinates, and generating a personnel risk embedding vector; s5, generating an updated risk thermodynamic diagram, calculating a risk score, and generating a risk early warning signal; and S6, triggering a physical device response according to the risk early warning signal, and collecting a device state feedback log. The problem that a traditional laboratory safety monitoring method is difficult to carry out early warning in combination with multi-source data can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of laboratory safety monitoring and management, and in particular to a smart laboratory safety monitoring method and system based on AIoT. Background Art

[0002] Traditional laboratory safety monitoring methods primarily rely on single-sensor threshold alarms or manual inspections, which suffer from defects such as isolated multi-source data and delayed warnings. This monitoring model not only struggles to capture potential risk factors in complex environments, but also fails to establish correlations between various safety factors, resulting in low overall system warning efficiency. Existing technologies utilize discrete and independent monitoring data for gases, equipment, and personnel, lacking real-time fusion and spatiotemporal alignment capabilities. Traditional threshold alarms can only respond passively after a risk occurs, and their fixed threshold settings are unable to adapt to the dynamic nature of the laboratory environment, making it impossible to achieve early warnings through multimodal data correlation analysis. Furthermore, due to the lack of intelligent risk assessment algorithms, the dynamic coordination mechanism between human behavior, equipment status, and environmental risks is insufficient, making it difficult to trigger graded warnings in a timely manner. Existing methods face technical bottlenecks in data fusion and feature alignment, particularly in extracting features from multi-source heterogeneous data and modeling their correlations, which restricts the proactive and reliable management of laboratory safety. Summary of the Invention

[0003] In view of this, the present invention aims to provide an AIoT-based smart laboratory safety monitoring method and system to solve the problem that traditional laboratory safety monitoring methods are difficult to combine multi-source data for early warning.

[0004] A smart laboratory safety monitoring method based on AIoT, including:

[0005] S1: Collect and preprocess multi-source heterogeneous data from the laboratory to obtain preprocessed gas concentration sequences, preprocessed thermal infrared image sequences, and preprocessed personnel video sequences;

[0006] S2: Extracting gas diffusion features based on the preprocessed gas concentration sequence, extracting thermal infrared features based on the preprocessed thermal infrared image sequence, and then calculating fusion features; the extraction of gas diffusion features includes generating dynamic convolution parameters and gating parameters based on wavelet decomposition and long short-term memory network, and performing multi-scale time series feature extraction on the gas concentration sequence through deformable convolution and gated weighting mechanism;

[0007] S3: Calculate spatial diffusion features based on gas diffusion features, then combine them with thermal infrared features to calculate physical constraint features and generate a risk heat map. The generation of the risk heat map includes extracting spatial diffusion features through deformable diffusion convolution, fusing thermal infrared features with spatial diffusion features through a spatiotemporal gating mechanism, and finally generating the risk heat map through dual-path decoding.

[0008] S4: Extracting person bounding boxes and person behavior features from the preprocessed person video sequence, calculating global risk coordinates, and generating a person risk embedding vector; generating the person risk embedding vector includes extracting person behavior features based on a dynamic threshold target detection network, performing perspective transformation coordinate mapping based on a risk heat map, and generating the person risk embedding vector through Gaussian weighting;

[0009] S5: Generate an updated risk heat map based on the personnel risk embedding vector and the risk heat map, calculate the risk score, and generate a risk warning signal;

[0010] S6: Trigger physical device responses based on risk warning signals and collect device status feedback logs.

[0011] Furthermore, the step S1 further includes:

[0012] S11: Collecting multi-source heterogeneous data in the laboratory, including: collecting gas concentration time series data through gas sensors; collecting thermal infrared image sequences through infrared thermal imagers; and collecting personnel video image sequences through surveillance cameras;

[0013] S12: For the gas concentration time series data, a sliding average filter is used to remove high-frequency noise, and the sensor baseline drift is eliminated by combining the Z-score normalization method to obtain the preprocessed gas concentration sequence;

[0014] S13: For the thermal infrared image sequence, non-uniformity correction is performed to eliminate the fixed noise of the thermal imager, and spatial alignment is performed to obtain a pre-processed thermal infrared image sequence;

[0015] S14: For the personnel video image sequence, a background subtraction method is used to extract the moving target, and then a morphological filtering process is performed to obtain a preprocessed personnel video sequence.

[0016] Furthermore, in the step S2, the extraction of thermal infrared features in the step S2 includes generating a physical constraint mask based on a discrete calculation result matrix of a partial differential equation of heat conduction, and applying the physical constraint mask to the convolution feature map through a Hadamard product to suppress activation responses that do not conform to the laws of thermodynamics; the calculation of the fusion features includes performing global average pooling of the thermal infrared features and aligning the dimensions of the multi-layer perceptron, and then realizing adaptive weighted fusion of gas diffusion features and thermal infrared features based on dynamic gating weights, and performing feature enhancement through layer normalization.

[0017] Furthermore, the step S2 further includes:

[0018] S21: Based on the preprocessed gas concentration sequence, calculate the dynamic convolution parameters and gating parameters, and extract the gas diffusion characteristics. The calculation method is:

[0019] Θ1,Θ2=LSTM(Wavelet(X gas ));

[0020]

[0021] Among them, Θ1 is the dynamic convolution parameter matrix, Θ2 is the gate parameter matrix, LSTM() is the long short-term memory network, Wavelet() is the wavelet decomposition, X gas is the gas concentration sequence after pretreatment, F gas is the gas diffusion feature, K is the number of parallel convolution kernels, k is the convolution kernel index, DCONV() is the deformable convolution, is the kth group of parameters in the dynamic convolution parameter matrix, is element-wise multiplication, is the kth group of parameters in the gating parameter matrix, sigmoid() is the sigmoid function;

[0022] S22: Generate a physical constraint mask based on the preprocessed thermal infrared image sequence and extract thermal infrared features. The calculation method is:

[0023]

[0024] F temp =ReLU(Conv(I)⊙exp(-λ·I″(x,y)));

[0025] Where I″(x,y) is the discrete calculation result matrix of the heat conduction partial differential equation, x, y are the image height coordinates and width coordinates in the preprocessed thermal infrared image sequence, I is the preprocessed thermal infrared image sequence, F temp is the thermal infrared feature, ReLU() is the ReLU function, Conv() is the convolutional layer, ⊙ is the Hadamard product, exp() is the exponential of the natural constant, λ is the thermal diffusion coefficient, and exp(-λ·I″(x,y)) is the physical constraint mask;

[0026] S23: Calculate the fusion characteristics based on the gas diffusion characteristics and thermal infrared characteristics. The calculation method is:

[0027] F′ temp =MLP(GAP(F temp ));

[0028] α=sigmoid(W α LayerNorm(F gas )+b α );

[0029] F fusion =α⊙F′temp +(1-α)⊙F gas ;

[0030] F′ fusion =LayerNorm(W fusion ·GeLU(F fusion )+b fusion );

[0031] Among them, F′ temp is the thermal infrared feature after dimension alignment, MLP() is the multi-layer perceptron, GAP() is the global average pooling, α is the dynamic fusion weight, W α is the dynamic fusion weight matrix, LayerNorm() is the layer normalization, b α is the dynamic fusion bias vector, F fusion is the initial fusion feature, F′ fusion is the fusion feature, W fusion is the fusion feature weight matrix, GeLU() is the GeLU function, b fusion is the fusion feature bias vector.

[0032] Furthermore, the dual-path decoding in the S3 step includes performing atrous convolution enhancement on the physical constraint features and then combining it with sigmoid function mapping, while implementing attention weighting on the fused features, and finally summing up to generate a risk heat map; the implementation of the spatiotemporal gating mechanism includes performing three-dimensional convolution splicing on the thermal infrared features and the spatial diffusion features, generating spatiotemporal weight coefficients, and forming physical constraint features through weighted fusion.

[0033] It should be further explained that, in order to solve the problem of feature fusion caused by the mismatch of spatiotemporal characteristics of laboratory multimodal data, the present invention achieves precise alignment of cross-modal features through dynamic parameter generation and physical constraint mechanism;

[0034] In gas diffusion feature extraction, the present invention adopts wavelet decomposition and long short-term memory network collaborative modeling, captures gas concentration mutation characteristics through frequency domain analysis, and adaptively adjusts multi-scale convolution kernel weights in combination with a temporal gating mechanism, solving the problem that traditional fixed convolution kernels cannot adapt to dynamic changes in gas diffusion rate. In thermal infrared feature extraction, a spatial constraint mask is generated based on the discretized calculation result matrix of the heat conduction partial differential equation, and the non-physical thermal radiation response is suppressed by an exponential decay function, overcoming the defect of conventional convolutional networks in insufficient modeling of thermodynamic laws. In the feature fusion stage, a dynamic gating weighting mechanism is designed to achieve dimensional alignment of cross-modal features through global feature compression and layer normalization processing, and a learnable weight coefficient is used to dynamically balance the contribution of the time-varying characteristics of gas diffusion and the spatial distribution of thermal radiation.

[0035] Compared with the traditional static weighted fusion method, the present invention uses differential equation constraints and adaptive parameter adjustment to make the fusion features accurately match the gas diffusion dynamics in the time domain and conform to the physical laws of heat conduction in the spatial domain, effectively improving the feature characterization capabilities of multi-source heterogeneous data in complex laboratory environments; especially for the characteristics of strong randomness of gas leakage direction in laboratory scenarios and significant interference of thermal imaging by equipment heat dissipation, through frequency-space-time three-dimensional joint modeling, a feature space covering the relationship between diffusion rate, heat conduction gradient and motion trajectory is constructed, which significantly enhances the system's early perception accuracy of sudden risk events.

[0036] Furthermore, the step S3 further includes:

[0037] S31: Based on the gas diffusion characteristics, the spatial diffusion characteristics are generated by deformable diffusion convolution. The calculation method is:

[0038]

[0039] Θ3=LSTM(Wavelet(F gas ));

[0040] Among them, F diff is the spatial diffusion feature, Θ3 is the spatial diffusion convolution kernel parameter matrix, is the residual connection, TimeDiff() is the time difference operator, Div() is the divergence operator, d is the diffusion coefficient, and SpatialGrad() is the spatial gradient operator;

[0041] S32: Based on the thermal infrared characteristics and spatial diffusion characteristics, the physical constraint characteristics are calculated through the spatiotemporal gating mechanism. The calculation method is:

[0042] G = sigmoid(Conv3D(Concat(F temp ,F diff )));

[0043] F phys =G⊙F temp +(1-G)⊙UpSample(F diff );

[0044] Among them, G is the spatiotemporal gating weight, Conv3D() is the three-dimensional convolution layer, Concat() is the splicing operation, and F phys It is a physical constraint feature, and UpSample() is a bilinear upsampling;

[0045] S33: Based on the fusion features and physical constraint features, dual-path decoding is performed to generate a risk heat map. The calculation method is:

[0046]

[0047] Among them, M risk is the risk heat map, DeConv() is the transposed convolution layer, W phys is the physical constraint weight matrix, * is the hole convolution, Attention() is the attention mechanism, W att is the fusion feature attention weight matrix.

[0048] It should be further explained that, in order to address the problem of insufficient accuracy in heat map generation caused by the complexity of laboratory risk transmission paths, the present invention dynamically generates deformable convolution kernel parameters based on wavelet components in spatial diffusion feature extraction, and constructs a partial differential equation enhancement model covering gas diffusion rate, concentration gradient and directionality through the joint constraints of differential operators of time difference and spatial gradient divergence, solving the problem of insufficient modeling of the physical laws of gas diffusion by traditional convolutional networks; in the fusion of physical constraint features, the present invention designs a spatiotemporal gating mechanism based on three-dimensional convolution, and generates dynamic weight coefficients that conform to the laws of thermodynamic conduction through cross-modal interactive learning of thermal imaging features and spatial diffusion features, overcoming the spatial dislocation problem caused by the simple superposition of thermal radiation and gas diffusion features in traditional methods; in the dual-path decoding stage, the receptive field of the physical constraint features is expanded through void convolution to capture the global risk characteristics, and then combined with the attention mechanism to enhance the response sensitivity of sudden leakage events, and finally form a high-resolution risk heat map;

[0049] Compared with the conventional single-path decoding method, the present invention realizes the coupled solution of the diffusion dynamics equation and the heat conduction equation based on mathematical principles, so that the risk thermodynamic map accurately reflects the gas diffusion process in the time domain and fits the distribution law of the heating area of ​​the equipment in the spatial domain. By integrating the dynamic parameter adjustment mechanism with the physical constraints, it effectively suppresses the false triggering of risk judgment due to normal heat dissipation of the equipment, and has stronger robustness in a multi-source interference environment.

[0050] Furthermore, the step S4 further includes:

[0051] S41: Based on the preprocessed person video sequence, the dynamic threshold target detection network is used to extract the person bounding box and person behavior features. The calculation method is:

[0052] B=YOLO v8 (V);

[0053] F people =Concat(STPN(B),OptFlow(V));

[0054] Among them, B is the person bounding box, YOLO v8 () is the YOLOv8 network, V is the pre-processed personnel video sequence, F peopleis the personnel behavior feature, STPN() is the spatiotemporal pyramid network, and OptFlow() is the optical flow motion feature extractor;

[0055] S42: Calculate the global risk coordinates based on the personnel bounding box. The calculation method is:

[0056]

[0057] in, are the horizontal and vertical coordinates of the global risk, Homography() is the perspective transformation, u and v are the horizontal and vertical coordinates of the center point of the personnel bounding box, T is the perspective transformation parameter, Width is the width of each video frame in the preprocessed personnel video sequence, Height is the height of each video frame in the preprocessed personnel video sequence, M risk (u,v) is the pixel value in the risk heat map, with the horizontal axis being u and the vertical axis being v;

[0058] S43: Generate the personnel risk embedding vector based on the global risk coordinates and personnel behavior characteristics. The calculation method is:

[0059]

[0060] Among them, E is the personnel risk embedding vector, W E Embedding weight matrix for personnel risk, In the risk heat map, the horizontal axis is The vertical axis is , μ1 and μ2 are the horizontal and vertical coordinates of the centroid of the risk heat map, and σ is the standard deviation of the pixel values ​​of the wind direction heat map.

[0061] Furthermore, the step S5 further includes:

[0062] S51: Generate an updated risk heat map based on the personnel risk embedding vector and the risk heat map. The calculation method is:

[0063] M′ risk =sigmoid(W M Concat(M riskl ,Expand(E))+b M );

[0064] Among them, M′ risk is the updated risk heat map, W M Update the weight matrix for the risk heat map, Expand() is the vector dimension expansion, b M Update the bias vector for the risk heatmap;

[0065] S52: Calculate the risk score based on the updated risk heat map. The calculation method is:

[0066] R=MaxPool(M′ risk )+GAP(M risk );

[0067] Among them, R is the risk score, MaxPool() is the global maximum pooling;

[0068] S53: Based on the risk score, a risk warning signal is generated through a hierarchical decision network. The calculation method is:

[0069]

[0070] Among them, Alert is a risk warning signal.

[0071] It should be further explained that, in response to the lack of coordinated consideration of personnel behavior and environmental risks, the present invention adaptively updates the risk heat map based on the personnel risk embedding vector. Through vector dimension expansion and nonlinear mapping of the weight matrix, the real-time location and behavior characteristics of personnel are spatially interacted with the heat map risk distribution, so that the heat map response value is significantly enhanced when personnel approach high-risk areas. At the same time, global maximum pooling is used to capture the spatial extreme value characteristics of sudden leakage events, and mean pooling is combined to retain the overall trend of the risk distribution.

[0072] Compared with the traditional single heat map analysis method, the present invention uses spatiotemporal coupling modeling of personnel behavior characteristics and risk diffusion to enable the system to perceive dynamic risk factors such as personnel entering dangerous areas and abnormal operations. In the complex environment of laboratory scenarios with frequent personnel flow and diverse operating behaviors, it can effectively improve the early warning response speed.

[0073] Furthermore, the step S6 further includes:

[0074] S61: According to the risk warning signal, the corresponding device combination is matched through the multi-level response rule library to obtain the target device activation list;

[0075] S62: triggering a device control instruction according to the target device activation list;

[0076] S63: According to the device control instruction, trigger the physical device response through the device control interface and collect the device status feedback log.

[0077] The present invention also discloses an AIoT-based smart laboratory safety monitoring system, comprising:

[0078] Multivariate data acquisition module: collects multi-source heterogeneous data from the laboratory and preprocesses it to obtain preprocessed gas concentration sequences, preprocessed thermal infrared image sequences, and preprocessed personnel video sequences;

[0079] Gas diffusion feature and thermal infrared feature fusion module: extracts gas diffusion features based on the preprocessed gas concentration sequence, extracts thermal infrared features based on the preprocessed thermal infrared image sequence, and then calculates the fusion features;

[0080] Risk heat map generation module: Calculates spatial diffusion characteristics based on gas diffusion characteristics, combines thermal infrared characteristics with physical constraint characteristics to generate a risk heat map;

[0081] Personnel risk embedding vector generation module: Based on the preprocessed personnel video sequence, it extracts the personnel bounding box and personnel behavior characteristics, calculates the global risk coordinates, and generates the personnel risk embedding vector;

[0082] Risk calculation module: generates an updated risk heat map based on the personnel risk embedding vector and risk heat map, calculates the risk score, and generates a risk warning signal;

[0083] Equipment control module: triggers physical equipment responses based on risk warning signals and collects equipment status feedback logs;

[0084] Compared with the prior art, the present invention has the following beneficial effects:

[0085] (1) The present invention constructs a multimodal data collaborative analysis framework to systematically solve the technical bottlenecks of isolated multi-source data and delayed warning in traditional laboratory safety monitoring. First, through dynamic convolution modeling of gas diffusion and thermal infrared physical mask constraints, accurate characterization of the time-varying characteristics of gas concentration and the spatial distribution of thermal radiation is achieved. Second, a spatiotemporal gating mechanism is introduced to embed the gas diffusion partial differential equation and the heat conduction law into the deep learning model to generate a risk heat map that conforms to the physical mechanism. Finally, the heat map is dynamically updated by embedding the personnel risk vector to achieve real-time coupling analysis of personnel behavior and risk propagation. Compared with the traditional single sensor threshold alarm method, this solution establishes a multimodal spatiotemporal alignment mechanism at the data layer and integrates physical laws and data-driven models at the feature layer to form a closed-loop management from risk perception to equipment linkage, significantly improving the timeliness of early warning.

[0086] (2) Aiming at the problem of multimodal data fusion in the laboratory, the present invention innovatively proposes a dynamic physical constraint feature extraction method. Through wavelet transform and long short-term memory network, the gas diffusion convolution kernel parameters are dynamically generated, and the feature weights are adaptively adjusted in combination with the timing gating mechanism, which solves the problem that the traditional fixed convolution kernel cannot adapt to the dynamic changes of gas diffusion. At the same time, a physical constraint mask is constructed based on the partial differential equation of heat conduction, which effectively suppresses the interference of equipment heat dissipation and significantly improves the system's early perception ability of sudden risk events in complex laboratory environments.

[0087] (3) Aiming at the characteristics of laboratory risk propagation, the present invention innovatively proposes a physical mechanism-guided risk heat map generation method, dynamically models the gas diffusion process, and combines the spatiotemporal gating mechanism to achieve accurate fusion of thermal radiation and diffusion characteristics, solving the problem of insufficient modeling of physical laws by traditional methods; the present invention proposes a dual-path decoding architecture, which captures the global risk pattern while enhancing the response sensitivity of local emergencies through the attention mechanism, thereby improving the accuracy of the risk heat map and effectively suppressing false alarms caused by equipment heat dissipation interference.

[0088] (4) The present invention dynamically updates the heat map by embedding the personnel risk vector, realizing real-time information interaction between personnel location and risk area. When personnel approach high-risk areas, the heat map response value is automatically enhanced. Combined with multi-scale pooling analysis, the early warning sensitivity of abnormal personnel behavior is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 A schematic diagram of a flow chart of a smart laboratory safety monitoring method based on AIoT provided by the present invention;

[0090] Figure 2 A schematic diagram of the algorithm flow for extracting gas diffusion characteristics provided by the present invention;

[0091] Figure 3 This is a schematic diagram of the algorithm flow for fusion feature extraction provided by the present invention. DETAILED DESCRIPTION

[0092] The present invention will be further described below with reference to the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention fall within the scope of protection of the present invention.

[0093] Example 1: A smart laboratory safety monitoring method based on AIoT, such as Figure 1 As shown, the following steps are included:

[0094] S1: Collect and preprocess multi-source heterogeneous data from the laboratory to obtain preprocessed gas concentration sequences, preprocessed thermal infrared image sequences, and preprocessed personnel video sequences;

[0095] S11: Collecting multi-source heterogeneous data in the laboratory, including: collecting time series data on the concentration of flammable gases (such as hydrogen and methane) and corrosive gases (such as hydrogen sulfide and hydrogen chloride) through gas sensors; collecting thermal infrared image sequences through infrared thermal imagers; and collecting personnel video image sequences through surveillance cameras;

[0096] S12: For the gas concentration time series data, a sliding average filter is used to remove high-frequency noise, and the sensor baseline drift is eliminated by combining the Z-score normalization method to obtain the preprocessed gas concentration sequence;

[0097] S13: For the thermal infrared image sequence, non-uniformity correction is performed to eliminate the fixed noise of the thermal imager, and spatial alignment is performed to obtain a pre-processed thermal infrared image sequence;

[0098] S14: For the personnel video image sequence, a background subtraction method is used to extract the moving target, and then a morphological filtering process is performed to obtain a preprocessed personnel video sequence.

[0099] S2: Extract gas diffusion features based on the preprocessed gas concentration sequence, extract thermal infrared features based on the preprocessed thermal infrared image sequence, and then calculate the fusion features;

[0100] S21: Based on the preprocessed gas concentration sequence, the dynamic convolution parameters and gating parameters are calculated, and the gas diffusion characteristics are extracted, such as Figure 2 As shown, the calculation method is:

[0101] Θ1,Θ2=LSTM(Wavelet(X gas ));

[0102]

[0103] Among them, Θ1 is the dynamic convolution parameter matrix, Θ2 is the gate parameter matrix, LSTM() is the long short-term memory network, Wavelet() is the wavelet decomposition, X gas is the gas concentration sequence after pretreatment, F gas is the gas diffusion feature, K is the number of parallel convolution kernels, k is the convolution kernel index, DCONV() is the deformable convolution, is the kth group of parameters in the dynamic convolution parameter matrix, is element-wise multiplication, is the kth group of parameters in the gating parameter matrix, sigmoid() is the sigmoid function;

[0104] S22: Generate a physical constraint mask based on the preprocessed thermal infrared image sequence and extract thermal infrared features. The calculation method is:

[0105]

[0106] F temp =ReLU(Conv(I)⊙exp(-λ·I″(x,y)));

[0107] Where I″(x,y) is the discrete calculation result matrix of the heat conduction partial differential equation, x, y are the image height coordinates and width coordinates in the preprocessed thermal infrared image sequence, I is the preprocessed thermal infrared image sequence, F temp is the thermal infrared feature, ReLU() is the ReLU function, Conv() is the convolutional layer, ⊙ is the Hadamard product, exp() is the exponential of the natural constant, λ is the thermal diffusion coefficient, and exp(-λ·I″(x,y)) is the physical constraint mask;

[0108] S23: Calculate fusion features based on gas diffusion features and thermal infrared features, such as Figure 3 As shown, the calculation method is:

[0109] F′ temp =MLP(GAP(F temp ));

[0110] α=sigmoid(W α LayerNorm(F gas )+b α );

[0111] F fusion =α⊙F′ temp +(1-α)⊙F gas ;

[0112] F′ fusion =LayerNorm(W fusion ·GeLU(F fusion )+b fusion );

[0113] Among them, F′ temp is the thermal infrared feature after dimension alignment, MLP() is the multi-layer perceptron, GAP() is the global average pooling, α is the dynamic fusion weight, W α is the dynamic fusion weight matrix, LayerNorm() is the layer normalization, b α is the dynamic fusion bias vector, F fusion is the initial fusion feature, F′ fusion is the fusion feature, W fusion is the fusion feature weight matrix, GeLU() is the GeLU function, b fusionis the fusion feature bias vector. Furthermore, the dual-path decoding in step S3 includes performing dilated convolution enhancement on the physical constraint features, combining it with sigmoid function mapping, and simultaneously applying attention weighting to the fused features, ultimately summing them to generate a risk heat map. The spatiotemporal gating mechanism also involves performing three-dimensional convolution on the thermal infrared features and spatial diffusion features to generate spatiotemporal weight coefficients, which are then weighted to form the physical constraint features.

[0114] S3: Calculate the spatial diffusion characteristics based on the gas diffusion characteristics, and then combine them with the thermal infrared characteristics to calculate the physical constraint characteristics and generate a risk heat map;

[0115] S31: Based on the gas diffusion characteristics, the spatial diffusion characteristics are generated by deformable diffusion convolution. The calculation method is:

[0116]

[0117] Θ3=LSTM(Wavelet(F gas ));

[0118] Among them, F diff is the spatial diffusion feature, Θ3 is the spatial diffusion convolution kernel parameter matrix, is the residual connection, TimeDiff() is the time difference operator, Div() is the divergence operator, d is the diffusion coefficient, and SpatialGrad() is the spatial gradient operator;

[0119] S32: Based on the thermal infrared characteristics and spatial diffusion characteristics, the physical constraint characteristics are calculated through the spatiotemporal gating mechanism. The calculation method is:

[0120] G = sigmoid(Conv3D(Concat(F temp ,F diff )));

[0121] F phys =G⊙F temp +(1-G)⊙UpSample(F diff );

[0122] Among them, G is the spatiotemporal gating weight, Conv3D() is the three-dimensional convolution layer, Concat() is the splicing operation, and F phys It is a physical constraint feature, and UpSample() is a bilinear upsampling;

[0123] S33: Based on the fusion features and physical constraint features, dual-path decoding is performed to generate a risk heat map. The calculation method is:

[0124]

[0125] Among them, M risk is the risk heat map, DeConv() is the transposed convolution layer, W phys is the physical constraint weight matrix, * is the hole convolution, Attention() is the attention mechanism, W att is the fusion feature attention weight matrix.

[0126] S4: Extract the person’s bounding box and behavior features based on the preprocessed person video sequence, calculate the global risk coordinates, and generate the person risk embedding vector;

[0127] S41: Based on the preprocessed person video sequence, the dynamic threshold target detection network is used to extract the person bounding box and person behavior features. The calculation method is:

[0128] B=YOLO v8 (V);

[0129] F people =Concat(STPN(B),OptFlow(V));

[0130] Among them, B is the person bounding box, YOLO v9 () is the YOLOv8 network, V is the pre-processed personnel video sequence, F people is the personnel behavior feature, STPN() is the spatiotemporal pyramid network, and OptFlow() is the optical flow motion feature extractor;

[0131] S42: Calculate the global risk coordinates based on the personnel bounding box. The calculation method is:

[0132]

[0133] in, are the horizontal and vertical coordinates of the global risk, Homography() is the perspective transformation, u and v are the horizontal and vertical coordinates of the center point of the personnel bounding box, T is the perspective transformation parameter, Width is the width of each video frame in the preprocessed personnel video sequence, Height is the height of each video frame in the preprocessed personnel video sequence, M risk (u,v) is the pixel value in the risk heat map, with the horizontal axis being u and the vertical axis being v;

[0134] S43: Generate the personnel risk embedding vector based on the global risk coordinates and personnel behavior characteristics. The calculation method is:

[0135]

[0136] Among them, E is the personnel risk embedding vector, W E Embedding weight matrix for personnel risk, In the risk heat map, the horizontal axis is The vertical axis is , μ1 and μ2 are the horizontal and vertical coordinates of the centroid of the risk heat map, and σ is the standard deviation of the pixel values ​​of the wind direction heat map.

[0137] S5: Generate an updated risk heat map based on the personnel risk embedding vector and the risk heat map, calculate the risk score, and generate a risk warning signal;

[0138] S51: Generate an updated risk heat map based on the personnel risk embedding vector and the risk heat map. The calculation method is:

[0139] M′ risk =sigmoid(W M Concat(M riskl ,Expand(E))+b M );

[0140] Among them, M′ risk is the updated risk heat map, W M Update the weight matrix for the risk heat map, Expand() is the vector dimension expansion, b M Update the bias vector for the risk heatmap;

[0141] S52: Calculate the risk score based on the updated risk heat map. The calculation method is:

[0142] R=MaxPool(M′ risk )+GAP(M risk );

[0143] Among them, R is the risk score, MaxPool() is the global maximum pooling;

[0144] S53: Based on the risk score, a risk warning signal is generated through a hierarchical decision network. The calculation method is:

[0145]

[0146] Among them, Alert is a risk warning signal.

[0147] In particular, for situations where regional risk characteristics vary significantly and equipment operation interference is complex, the present invention also provides an adaptive risk warning signal calculation method to replace the calculation formula in S53, so as to enhance the adaptive warning capability of the smart laboratory safety monitoring system for different risk scenarios, effectively reducing the false alarm rate while ensuring a high detection rate. Specifically,

[0148]

[0149] c=MLP(Conv(M′ risk ));

[0150] Among them, Alert is the risk warning signal, and c is the risk threshold coefficient.

[0151] S6: Trigger physical device responses based on risk warning signals and collect device status feedback logs;

[0152] S61: According to the risk warning signal, the corresponding device combination is matched through the multi-level response rule library to obtain the target device activation list;

[0153] S62: triggering a device control instruction according to the target device activation list;

[0154] S63: According to the device control instruction, trigger the physical device response through the device control interface and collect the device status feedback log.

[0155] For example, if S53 calculates the risk warning signal Alert=2 (high risk), and the laboratory equipment status is: ventilation system standby, sprinkler device off, and sound and light alarm standby, then:

[0156] Device matching stage:

[0157] Input: Alert=2;

[0158] Processing: Query the multi-level response rule base (secondary warning, activation of ventilation system + sound and light alarm + sprinkler device);

[0159] Output: Target equipment activation list = [ventilation system (power 80%), sound and light alarm (continuous mode), sprinkler (delayed start 5 seconds)];

[0160] During the instruction generation phase, the device control instruction sequence is generated as follows:

[0161] Immediately trigger the sound and light alarm to flash red (frequency 2Hz);

[0162] Start the ventilation system (target power 80%, gradually accelerating);

[0163] Send the sprinkler preparation command (5 seconds delay start);

[0164] Output: Device control instruction set = {"device":"alarm","mode":"blink_red","freq":2},

[0165] {"device":"ventilation","power":80,"ramp":5},

[0166] {"device":"sprinkler","delay":5}

[0167] };

[0168] During the equipment execution phase, control signals are sent via the Modbus-RTU protocol for real-time monitoring:

[0169] Sound and light alarm status feedback (confirmation of activation);

[0170] Actual ventilation system power (currently 65%, approaching 80% at a rate of 3% / s);

[0171] Spray device countdown status (3.2 seconds remaining);

[0172] Output: Device status log = {

[0173] "timestamp":"2023-05-20T14:23:45",

[0174] "alarm":"active",

[0175] "ventilation":{"target":80,"current":65},

[0176] "sprinkler":{"status":"preheated","countdown":3.2}

[0177] };

[0178] Example 2: The present invention also discloses an AIoT-based smart laboratory safety monitoring system, comprising:

[0179] Multivariate data acquisition module: collects multi-source heterogeneous data from the laboratory and preprocesses it to obtain preprocessed gas concentration sequences, preprocessed thermal infrared image sequences, and preprocessed personnel video sequences;

[0180] Gas diffusion feature and thermal infrared feature fusion module: extracts gas diffusion features based on the preprocessed gas concentration sequence, extracts thermal infrared features based on the preprocessed thermal infrared image sequence, and then calculates the fusion features;

[0181] Risk heat map generation module: Calculates spatial diffusion characteristics based on gas diffusion characteristics, combines thermal infrared characteristics with physical constraint characteristics to generate a risk heat map;

[0182] Personnel risk embedding vector generation module: Based on the preprocessed personnel video sequence, it extracts the personnel bounding box and personnel behavior characteristics, calculates the global risk coordinates, and generates the personnel risk embedding vector;

[0183] Risk calculation module: generates an updated risk heat map based on the personnel risk embedding vector and risk heat map, calculates the risk score, and generates a risk warning signal;

[0184] Equipment control module: triggers physical equipment responses based on risk warning signals and collects equipment status feedback logs.

[0185] It should be noted that the serial numbers of the above-mentioned embodiments of the present invention are for descriptive purposes only and do not represent the advantages or disadvantages of the embodiments. In addition, the terms "including", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method comprising the element.

[0186] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in multiple embodiments of the present invention.

[0187] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A smart laboratory safety monitoring method based on AIoT, characterized in that: The following steps are involved: S1: Collect and preprocess multi-source heterogeneous data from the laboratory to obtain preprocessed gas concentration sequences, preprocessed thermal infrared image sequences, and preprocessed personnel video sequences; S2: extracting gas diffusion features based on the preprocessed gas concentration sequence, extracting thermal infrared features based on the preprocessed thermal infrared image sequence, and then calculating fusion features; The extraction of gas diffusion characteristics includes generating dynamic convolution parameters and gating parameters based on wavelet decomposition and long short-term memory network, and extracting multi-scale time series characteristics of gas concentration sequence through deformable convolution and gating weighting mechanism; S3: Calculate spatial diffusion features based on gas diffusion features, then combine them with thermal infrared features to calculate physical constraint features and generate a risk heat map. The generation of the risk heat map includes extracting spatial diffusion features through deformable diffusion convolution, fusing thermal infrared features with spatial diffusion features through a spatiotemporal gating mechanism, and finally generating the risk heat map through dual-path decoding. S4: Extract the person’s bounding box and behavior features based on the preprocessed person video sequence, calculate the global risk coordinates, and generate the person risk embedding vector; The generation of the personnel risk embedding vector includes extracting personnel behavior features based on a dynamic threshold target detection network, performing perspective transformation coordinate mapping based on a risk heat map, and generating the personnel risk embedding vector through Gaussian weighting; S5: Generate an updated risk heat map based on the personnel risk embedding vector and the risk heat map, calculate the risk score, and generate a risk warning signal; S6: Trigger physical device responses based on risk warning signals and collect device status feedback logs.

2. The AIoT-based smart laboratory safety monitoring method according to claim 1 is characterized in that: The S1 step includes: S11: Collecting multi-source heterogeneous data in the laboratory, including: collecting gas concentration time series data through gas sensors; collecting thermal infrared image sequences through infrared thermal imagers; and collecting personnel video image sequences through surveillance cameras; S12: For the gas concentration time series data, a sliding average filter is used to remove high-frequency noise, and the sensor baseline drift is eliminated by combining the Z-score normalization method to obtain the preprocessed gas concentration sequence; S13: For the thermal infrared image sequence, non-uniformity correction is performed to eliminate the fixed noise of the thermal imager, and spatial alignment is performed to obtain a pre-processed thermal infrared image sequence; S14: For the personnel video image sequence, a background subtraction method is used to extract the moving target, and then a morphological filtering process is performed to obtain a preprocessed personnel video sequence.

3. The AIoT-based smart laboratory safety monitoring method according to claim 1, characterized in that: Extracting the thermal infrared features in step S2 includes generating a physical constraint mask based on a discrete calculation result matrix of a heat conduction partial differential equation, and applying the physical constraint mask to the convolution feature map through a Hadamard product to suppress activation responses that do not conform to thermodynamic laws; The calculation of the fusion features includes performing global average pooling on the thermal infrared features and aligning the dimensions of the multi-layer perceptron, realizing adaptive weighted fusion of the gas diffusion features and the thermal infrared features based on dynamic gating weights, and performing feature enhancement through layer normalization.

4. The AIoT-based smart laboratory safety monitoring method according to claim 3 is characterized in that: The S2 step comprises: S21: Based on the preprocessed gas concentration sequence, calculate the dynamic convolution parameters and gating parameters, and extract the gas diffusion characteristics. The calculation method is: Θ1,Θ2=LSTM(Wavelet(X gas )); Among them, Θ1 is the dynamic convolution parameter matrix, Θ2 is the gate parameter matrix, LSTM() is the long short-term memory network, Wavelet() is the wavelet decomposition, X gas is the gas concentration sequence after pretreatment, F gas is the gas diffusion feature, K is the number of parallel convolution kernels, k is the convolution kernel index, DCONV() is the deformable convolution, is the kth group of parameters in the dynamic convolution parameter matrix, is element-wise multiplication, is the kth group of parameters in the gating parameter matrix, sigmoid() is the sigmoid function; S22: Generate a physical constraint mask based on the preprocessed thermal infrared image sequence and extract thermal infrared features. The calculation method is: F temp =ReLU(Conv(I)⊙exp(-λ·I″(x,y))); Where I″(x,y) is the discrete calculation result matrix of the heat conduction partial differential equation, x, y are the image height coordinates and width coordinates in the preprocessed thermal infrared image sequence, I is the preprocessed thermal infrared image sequence, F temp is the thermal infrared feature, ReLU() is the ReLU function, Conv() is the convolutional layer, ⊙ is the Hadamard product, exp() is the exponential of the natural constant, λ is the thermal diffusion coefficient, and exp(-λ·I″(x,y)) is the physical constraint mask; S23: Calculate the fusion characteristics based on the gas diffusion characteristics and thermal infrared characteristics. The calculation method is: F′ temp =MLP(GAP(F temp )); α=sigmoid(W α ·LayerNorm(F gas )+b α ); F fusion =α⊙F′ temp +(1-α)⊙F gas ; F′ fusion =LayerNorm(W fusion ·GeLU(F fusion )+b fusion ); Among them, F′ temp is the thermal infrared feature after dimension alignment, MLP() is the multi-layer perceptron, GAP() is the global average pooling, α is the dynamic fusion weight, W α is the dynamic fusion weight matrix, LayerNorm() is the layer normalization, b α is the dynamic fusion bias vector, F fusion is the initial fusion feature, F′ fusion is the fusion feature, W fusion is the fusion feature weight matrix, GeLU() is the GeLU function, b fusion is the fusion feature bias vector.

5. The AIoT-based smart laboratory safety monitoring method according to claim 4 is characterized in that: The dual-path decoding in the S3 step includes performing dilated convolution enhancement on the physical constraint features and then combining it with sigmoid function mapping, while implementing attention weighting on the fused features, and finally summing up to generate a risk heat map; the implementation of the spatiotemporal gating mechanism includes performing three-dimensional convolution splicing on the thermal infrared features and spatial diffusion features, generating spatiotemporal weight coefficients, and forming physical constraint features through weighted fusion.

6. The AIoT-based smart laboratory safety monitoring method according to claim 5 is characterized in that: The S3 step includes: S31: Based on the gas diffusion characteristics, the spatial diffusion characteristics are generated by deformable diffusion convolution. The calculation method is: Θ3=LSTM(Wavelet(F gas )); Among them, F diff is the spatial diffusion feature, Θ3 is the spatial diffusion convolution kernel parameter matrix, is the residual connection, TimeDiff() is the time difference operator, Div() is the divergence operator, d is the diffusion coefficient, and SpatialGrad() is the spatial gradient operator; S32: Based on the thermal infrared characteristics and spatial diffusion characteristics, the physical constraint characteristics are calculated through the spatiotemporal gating mechanism. The calculation method is: G=sigmoid(Conv3D(Concat(F temp ,F diff ))); F phys =G⊙F temp +(1-G)⊙UpSample(F diff ); Among them, G is the spatiotemporal gating weight, Conv3D() is the three-dimensional convolution layer, Concat() is the splicing operation, and F phys It is a physical constraint feature, and UpSample() is a bilinear upsampling; S33: Based on the fusion features and physical constraint features, dual-path decoding is performed to generate a risk heat map. The calculation method is: Among them, M risk is the risk heat map, DeConv() is the transposed convolution layer, W phys is the physical constraint weight matrix, ★ is the dilated convolution, Attention() is the attention mechanism, W att is the fusion feature attention weight matrix.

7. The AIoT-based smart laboratory safety monitoring method according to claim 6, characterized in that: The S4 step comprises: S41: Based on the preprocessed person video sequence, the dynamic threshold target detection network is used to extract the person bounding box and person behavior features. The calculation method is: B=YOLO v8 (V); F people =Concat(STPN(B),OptFlow(V)); Among them, B is the person bounding box, YOLO v8 () is the YOLOv8 network, V is the pre-processed personnel video sequence, F people is the personnel behavior feature, STPN() is the spatiotemporal pyramid network, and OptFlow() is the optical flow motion feature extractor; S42: Calculate the global risk coordinates based on the personnel bounding box. The calculation method is: in, are the horizontal and vertical coordinates of the global risk, Homography() is the perspective transformation, u and v are the horizontal and vertical coordinates of the center point of the personnel bounding box, T is the perspective transformation parameter, Width is the width of each video frame in the preprocessed personnel video sequence, Height is the height of each video frame in the preprocessed personnel video sequence, M risk (u,v) is the pixel value in the risk heat map, with the horizontal axis being u and the vertical axis being v; S43: Generate the personnel risk embedding vector based on the global risk coordinates and personnel behavior characteristics. The calculation method is: Among them, E is the personnel risk embedding vector, W E Embedding weight matrix for personnel risk, In the risk heat map, the horizontal axis is The vertical axis is , μ1 and μ2 are the horizontal and vertical coordinates of the centroid of the risk heat map, and σ is the standard deviation of the pixel values ​​of the wind direction heat map.

8. The AIoT-based smart laboratory safety monitoring method according to claim 7, characterized in that: The step S5 comprises: S51: Generate an updated risk heat map based on the personnel risk embedding vector and the risk heat map. The calculation method is: M′ risk =sigmoid(W M ·Concat(M riskl ,Expand(E))+b M ); Among them, M′ risk is the updated risk heat map, W M Update the weight matrix for the risk heat map, Expand() is the vector dimension expansion, b M Update the bias vector for the risk heatmap; S52: Calculate the risk score based on the updated risk heat map. The calculation method is: R=MaxPool(M′ risk )+GAP(M ri3k ); Among them, R is the risk score, MaxPool() is the global maximum pooling; S53: Based on the risk score, a risk warning signal is generated through a hierarchical decision network. The calculation method is: Among them, Alert is a risk warning signal.

9. The AIoT-based smart laboratory safety monitoring method according to claim 8, characterized in that: The step S6 comprises: S61: According to the risk warning signal, the corresponding device combination is matched through the multi-level response rule library to obtain the target device activation list; S62: triggering a device control instruction according to the target device activation list; S63: According to the device control instruction, trigger the physical device response through the device control interface and collect the device status feedback log.

10. A smart laboratory safety monitoring system based on AIoT, characterized by: include: Multivariate data acquisition module: collects multi-source heterogeneous data from the laboratory and preprocesses it to obtain preprocessed gas concentration sequences, preprocessed thermal infrared image sequences, and preprocessed personnel video sequences; Gas diffusion feature and thermal infrared feature fusion module: extracts gas diffusion features based on the preprocessed gas concentration sequence, extracts thermal infrared features based on the preprocessed thermal infrared image sequence, and then calculates the fusion features; Risk heat map generation module: Calculates spatial diffusion characteristics based on gas diffusion characteristics, combines thermal infrared characteristics with physical constraint characteristics to generate a risk heat map; Personnel risk embedding vector generation module: Based on the preprocessed personnel video sequence, it extracts the personnel bounding box and personnel behavior characteristics, calculates the global risk coordinates, and generates the personnel risk embedding vector; Risk calculation module: generates an updated risk heat map based on the personnel risk embedding vector and risk heat map, calculates the risk score, and generates a risk warning signal; Equipment control module: triggers physical equipment responses based on risk warning signals and collects equipment status feedback logs; To realize an AIoT-based smart laboratory safety monitoring method as described in any one of claims 1-9.

Images