Intelligent construction safety monitoring system and method based on multi-modal data fusion

Abstract

The invention relates to the technical field of safety monitoring of constructional engineering, in particular to an intelligent construction safety monitoring system and method based on multi-modal data fusion, and the system comprises a plurality of modules: a 5GMEC-based heterogeneous data space-time alignment module which completes coordinate system conversion between laser point cloud and a BIM model by using an improved Fast-ICP algorithm; the feature level fusion network comprises a geometric feature branch (improved unscented Kalman filter processing point cloud normal vector) and a time sequence feature branch (LSTM processing stress sensor data); the NeRFXL fusion engine is used for fusing the point cloud and the video data by adopting a multi-scale neural radiation field; the multi-task anomaly detector is used for constructing a hierarchical Transform architecture and introducing a modal gating mechanism; and the dynamic knowledge graph engine constructs a knowledge graph updating module based on a graph neural network, and all the modules are in communication connection with one another and work cooperatively, so that functions of data processing, fusion, anomaly detection, knowledge updating and the like are realized.

Description

Technical Field

[0001] This invention relates to the field of building engineering safety monitoring technology, and in particular to an intelligent construction safety monitoring system and method based on multimodal data fusion, which enables real-time monitoring, early warning and decision-making regarding the safety status of construction sites. Background Technology

[0002] With the increasing scale and complexity of modern construction projects, construction safety monitoring has become a crucial link in ensuring project quality and personnel safety. Traditional construction safety monitoring methods mainly rely on single types of data sources, such as manual inspections, single-point sensor monitoring, or video surveillance, which suffer from problems such as limited coverage, information silos, and delayed response.

[0003] Current construction safety monitoring systems on the market have significant shortcomings in terms of multi-source data fusion accuracy, anomaly detection efficiency, and dynamic decision support. Traditional methods typically suffer from registration errors between laser point clouds and BIM models exceeding ±5mm, cross-modal feature association accuracy below 85%, and end-to-end system latency exceeding 500ms. Existing technologies rely on single-sensor data and lack multi-modal collaborative analysis capabilities, resulting in anomaly detection false alarm rates as high as 30%. Furthermore, traditional knowledge graphs are outdated and unable to respond in real-time to changes in construction scenarios. Emergency decision-making relies on human experience, with response times exceeding 2 seconds, failing to meet the safety monitoring needs of modern large-scale and complex projects. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent construction safety monitoring system and method for multimodal data fusion, aiming to solve the technical problems existing in the prior art, such as low accuracy of multi-source data fusion, poor efficiency of anomaly detection, and lag in system response.

[0005] This invention proposes an intelligent construction safety monitoring system based on multimodal data fusion, comprising:

[0006] The heterogeneous data spatiotemporal alignment module based on 5GMEC is used to realize the coordinate system transformation between laser point cloud and BIM model through the improved Fast-ICP algorithm, and to establish a rigid transformation matrix T=[R|t], where: R is the rotation matrix, satisfying R∈SO(3), SO(3) is a three-dimensional special orthogonal group; t is the translation vector, satisfying t∈R 3 R 3 Represents the three-dimensional real number space;

[0007] A feature-level fusion network, communicatively connected to the heterogeneous data spatiotemporal alignment module, is used to construct a two-stream depthwise separable convolutional network, including a geometric feature branch and a temporal feature branch. The geometric feature branch uses an improved unscented Kalman filter to process the point cloud normal vectors, and the calculation formula is as follows:

[0008]

[0009] Where: n i p is the normal vector of the i-th point; i Let be the 3D coordinate vector of the i-th point in the point cloud; × represents the cross product of vectors; |·| represents the magnitude of the vector.

[0010] The time-series feature branch processes stress sensor data using LSTM, and the calculation formula is as follows:

[0011] h t =σ(W h [h t-1 ,x t ]+b h ),

[0012] Where: h t h is the hidden state vector at time t; t-1 Let x be the hidden state vector at time t-1; t W is the input vector at time t; h b is the weight matrix; h σ is the bias vector; σ is the activation function; [·,·] denotes vector concatenation operation;

[0013] The NeRFXL fusion engine, which communicates with the feature-level fusion network, is used to fuse point cloud and video data using multi-scale neural radiation fields, and to define the radiation field function:

[0014] C(r)=∫T(t)σ(x(t))c(x(t),d)dt,

[0015] Where: C(r) is the color value of light r; T(t) is the transmittance; σ(x(t)) is the volume density at position x(t); c(x(t),d) is the color value at position x(t) along direction d; ∫ represents the integral operation;

[0016] A multi-task anomaly detector, communicating with the NeRFXL fusion engine, is used to construct a hierarchical Transformer architecture. A modal gating mechanism is introduced in the self-attention layer, and the attention calculation formula is as follows:

[0017]

[0018] Where: Q is the query matrix; K is the key matrix; V is the value matrix; d k is the dimension of the key vector; G is the modal gating matrix with elements in the interval [0,1]; ⊙ represents element-wise multiplication; softmax is the normalized exponential function;

[0019] The dynamic knowledge graph engine, which communicates with the multi-task anomaly detector, is used to construct a knowledge graph update module based on a graph neural network and defines a relation propagation function.

[0020]

[0021] in: Here is the feature representation of node v at layer l+1; W represents the feature representation of node u at layer l. (l) b is the weight matrix of the l-th layer; (l) σ is the bias vector of the l-th layer; N(v) is the set of neighboring nodes of node v; σ is the activation function; ∑ represents the summation operation.

[0022] Preferably, the heterogeneous data spatiotemporal alignment module includes:

[0023] A two-stage clock synchronization unit is used to implement the hardware-level IEEE 1588 precise time protocol and the software-level NTP time synchronization.

[0024] The coordinate system unit is used to establish the global coordinate system transformation tree and define the transformation relationships:

[0025]

[0026] Where: x g x is the coordinate vector in the global coordinate system; local It is a coordinate vector in the local coordinate system; A 4×4 homogeneous transformation matrix containing rotation and translation transformations; · denotes matrix multiplication; the objective function optimization unit is used to establish the registration objective function:

[0027]

[0028] Where: E(R,t) is the error function; R is the rotation matrix; t is the translation vector; p i Let q be the coordinates of the i-th point in the source point cloud; i The coordinates of the corresponding point in the target point cloud; N is the total number of matching point pairs; |·| 2 ∑ represents the squared Euclidean distance; ∑ represents the summation operation.

[0029] Preferably, the feature-level fusion network includes:

[0030] The geometric feature extraction unit processes point cloud data using an improved PointNet++ architecture, extracting point cloud features through a multilayer perceptron. The calculation formula is as follows:

[0031] f θ (p i )=σ(W (2) (σ(W(1) p i +b (1) ))+b (2) ),

[0032] Where: f θ (p i Let p be a point. i Feature representation; W (1) W is the first layer weight matrix; (2) b is the weight matrix for the second layer; (1) b is the first layer bias vector; (2) σ is the second-layer bias vector; σ is the activation function.

[0033] The temporal feature extraction unit processes sensor data using a bidirectional LSTM and employs a temporal attention mechanism. The calculation formula is as follows:

[0034] α t =softmax(v T tanh(W h h t +W s s)),

[0035] Where: α t Let be the attention weight at time t; v be the learnable attention vector; W ... h h is the hidden state transformation matrix; t Let W be the hidden state at time t; s s is the global context transformation matrix; s is the global context vector;

[0036] tanh is the hyperbolic tangent activation function; the feature association unit is used to establish the cross-modal covariance matrix, and its calculation formula is:

[0037] Σ XY =E[(X-μ)] X )(Y-μ Y ) T ],

[0038] Wherein: Σ XY X is the covariance matrix; Y is the geometric eigenvector; μ is the temporal eigenvector; X is the geometric eigenvector; μ is the temporal eigenvector; μ is the ... X Let μ be the mean vector of X; Y Let Y be the mean vector; E[·] represents the expected value; (·) T This indicates the matrix transpose.

[0039] Preferably, the NeRFXL fusion engine includes:

[0040] Spatial hash coding unit, used to map input coordinates to a multi-resolution hash table, is calculated using the following formula:

[0041] γ(x)=[h1(x),...,h L (x)],

[0042] Where: γ(x) is the hash code vector of coordinate x; h L (x) is the L-th level hash function, which maps coordinates to hash table indices; L is the hash level; [·,…,·] represents vector concatenation operation;

[0043] Dynamic density modeling elements are used to define deformable radiation field functions, and the calculation formula is as follows:

[0044] σ(x,δ)=MLP(γ(x+Δ(x,δ))),

[0045] Where: σ(x,δ) is the volume density at position x; δ is the deformation parameter, such as the displacement of the support structure; Δ(x,δ) is the deformation field function; MLP represents a multilayer perceptron, which consists of multiple fully connected layers;

[0046] Real-time rendering unit for supporting interactive viewpoint rendering at 4096×2160@30fps.

[0047] Preferably, the multi-task anomaly detector includes:

[0048] Modality-specific coding units (MSCs) are used to design independent position codes for each mode, and their calculation formula is as follows:

[0049] PE(pos,2i)=sin(pos / 10000 2i / d ),

[0050] Where: PE is the location code; pos is the spatiotemporal location index; i is the coding dimension index; d is the total dimension of the location code; sin is the sine function;

[0051] A gated cross-attention unit is used to compute cross-modal attention, as described in claim 1;

[0052] A three-dimensional spatiotemporal attention unit is used to construct a joint spatial-temporal attention mechanism. The calculation formula is as follows:

[0053]

[0054] Where: H is the spatial dimension, representing the voxel grid size; T is the time series length; q i Let k be the query vector for the i-th spatial location; j v is the key vector at the j-th time step; j Let be the value vector at the j-th time step; d is the vector dimension; exp is the exponential function;

[0055] The mixed density scoring unit is used to calculate the probability of anomalies, and the calculation formula is as follows:

[0056]

[0057] Where: p(y) is the outlier probability distribution; K is the Gaussian mixture component number; π k μ is the mixing coefficient of the k-th Gaussian distribution; k Let Σ be the mean vector of the k-th distribution; k Let N(·|·,·) be the covariance matrix of the k-th distribution; N(·|·,·) represents the probability density function of the multivariate Gaussian distribution.

[0058] Preferably, the dynamic knowledge graph engine includes:

[0059] Safety ontology definition unit, used to construct the construction safety ontology:

[0060]

[0061] Where: O is the ontology; ε is the set of entities; Let A be the set of relations; and let A be the set of axiom constraints.

[0062] The spectrum update unit is used to implement event-driven spectrum updates. The spectrum update is triggered when the detected stress value meets the following conditions:

[0063] σ t >μ+3δ,

[0064] Where: σ t denoted as t, where μ is the stress value detected at time t; μ is the mean of historical stress data; and δ is the standard deviation of historical stress data.

[0065] Heterogeneous graph attention network unit, used to define node update formula;

[0066]

[0067] in: Let K be the feature representation of node v at layer l+1; K is the number of attention heads. W represents the attention weight of the node pair (u,v) in the k-th attention head; k Let be the weight matrix for the k-th attention head; Let be the feature representation of node u at layer l; N(v) be the set of neighboring nodes of node v; σ be the activation function; || denotes the multi-head attention concatenation operation;

[0068] The multi-hop inference unit is used to calculate the probability of a relationship; the calculation formula is as follows:

[0069] P(h|r,t)=σ(h T·diag(r)·t),

[0070] Where: P(h|r,t) is the probability that a relation r exists between a given head entity h and a tail entity t; diag(r) is the matrix formed by diagonalizing the relation vector r; σ is the sigmoid activation function, which maps the output to the interval [0,1]; (·) T This indicates the transpose operation.

[0071] Preferably, the spatial hash encoding unit divides the three-dimensional space into 16 resolution levels, with each level's spatial resolution ranging from 2... 2 to 2 16 .

[0072] Preferably, a real-time early warning module is also included, which is communicatively connected to the dynamic knowledge graph engine and the multi-task anomaly detector, for:

[0073] When the displacement value displayed in the 3D heatmap generated by the NeRF engine exceeds the threshold, a yellow warning signal is generated;

[0074] The 5G network will push disposal plans to terminal devices, including stopping excavation, adding support, and starting dewatering.

[0075] Preferably, the three-dimensional spatiotemporal attention unit divides the point cloud data into a 64x64 grid partition and divides the sensor time-series data into 60-second intervals to achieve parallel attention calculation for 8 heads.

[0076] Intelligent construction safety monitoring methods based on multimodal data fusion include:

[0077] The spatiotemporal registration of laser point clouds and BIM models is achieved using an improved Fast-ICP algorithm. The rigid transformation matrix T = [R|t] is solved, where: R is the rotation matrix, satisfying R∈SO(3), SO(3) is a three-dimensional special orthogonal group; t is the translation vector, satisfying t∈R 3 R 3 Represents the three-dimensional real number space;

[0078] A dual-stream depthwise separable convolutional network is used to simultaneously process point cloud normal vectors and stress sensor temporal data. The formula for calculating point cloud normal vectors is as follows:

[0079]

[0080] Where: n i p is the normal vector of the i-th point; i Let be the 3D coordinate vector of the i-th point in the point cloud; × represents the cross product of vectors; |·| represents the magnitude of the vector.

[0081] A real-time 3D heatmap with a resolution of 4096×2160 was generated using a multi-scale neural radiation field fusion engine. The radiation field function is:

[0082] C(r)=∫T(t)σ(x(t))c(x(t),d)dt,

[0083] Where: C(r) is the color value of light r; T(t) is the transmittance; σ(x(t)) is the volume density at position x(t); c(x(t),d) is the color value at position x(t) along direction d; ∫ represents the integral operation;

[0084] Cross-modal attention weights are calculated using a hierarchical Transformer architecture. The attention calculation formula is as follows:

[0085]

[0086] Where: Q is the query matrix; K is the key matrix; V is the value matrix; d k is the dimension of the key vector; G is the modal gating matrix with elements in the interval [0,1]; ⊙ represents element-wise multiplication; softmax is the normalized exponential function;

[0087] The node update formula for dynamically updating the construction safety knowledge graph based on a graph neural network is as follows:

[0088]

[0089] in: Here is the feature representation of node v at layer l+1; W represents the feature representation of node u at layer l. (l) b is the weight matrix of the l-th layer; (l) σ is the bias vector of the l-th layer; N(v) is the set of neighboring nodes of node v; σ is the activation function; ∑ represents the summation operation.

[0090] Specifically, the beneficial effects of the present invention include:

[0091] 1. Improved accuracy of multi-source data fusion: Point cloud registration error ≤ ±1.5mm (traditional method ±5mm), cross-modal feature association accuracy reaches 98.7%;

[0092] 2. Improved anomaly detection efficiency: 3D defect recognition speed of 120 frames / second, and multimodal anomaly detection F1-score of 0.927;

[0093] 3. System response performance optimization: end-to-end latency <200ms (under 5G network), knowledge graph inference response time <500ms;

[0094] 4. Predictive maintenance capability: Provides early warning of potential risks exceeding limits 10 minutes in advance and automatically pushes out disposal solutions. Attached Figure Description

[0095] Figure 1 The system architecture diagram of this invention shows the main modules of the five-layer progressive architecture and their connection relationships;

[0096] Figure 2 This is a schematic diagram of the spatiotemporal alignment module for heterogeneous data based on 5GMEC.

[0097] Figure 3 This is a schematic diagram of a two-stream architecture for a feature-level fusion network.

[0098] Figure 4 A multi-scale neural radiation field structure diagram of the NeRFXL fusion engine;

[0099] Figure 5 A hierarchical Transformer architecture diagram for a multi-task anomaly detector;

[0100] Figure 6 A flowchart for updating the graph neural network in a dynamic knowledge graph engine;

[0101] Figure 7 This is an example diagram illustrating the system's application in real-time monitoring of foundation pit construction.

[0102] Figure 8 This is an example diagram illustrating the application of the system in the quality inspection of shield tunnel segment assembly. Detailed Implementation

[0103] Please refer to the attached document. Figure 1-8 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0104] like Figure 1 As shown, the intelligent construction safety monitoring system for multimodal data fusion provided by the present invention adopts a five-layer progressive architecture, including: a heterogeneous data spatiotemporal alignment module 10 based on 5GMEC, a feature-level fusion network 20, a NeRFXL fusion engine 30, a multi-task anomaly detector 40, and a dynamic knowledge graph engine 50.

[0105] The heterogeneous data spatiotemporal alignment module 10 based on 5GMEC realizes the coordinate system transformation between laser point cloud and BIM model through the improved Fast-ICP algorithm, and establishes a rigid transformation matrix T = [R|t]. Here, R is the rotation matrix, satisfying R∈SO(3), where SO(3) is a three-dimensional special orthogonal group; t is the translation vector, satisfying t∈R 3 R 3It represents a three-dimensional real number space. This module solves the spatiotemporal alignment problem of multi-source heterogeneous data, laying the foundation for subsequent fusion analysis.

[0106] The feature-level fusion network 20 is communicatively connected to the heterogeneous data spatiotemporal alignment module 10 to construct a two-stream depthwise separable convolutional network, including a geometric feature branch and a temporal feature branch. The geometric feature branch uses an improved unscented Kalman filter (UKF) to process the point cloud normal vectors, while the temporal feature branch processes stress sensor data using LSTM. Through cross-modal feature association, effective fusion of point cloud geometric features and sensor temporal features is achieved.

[0107] The NeRFXL fusion engine 30 communicates with the feature-level fusion network 20, employing multi-scale neural radiation field technology to fuse point cloud and video data. This engine defines a radiation field function, enabling high-resolution 3D scene reconstruction and real-time rendering, providing intuitive visualization support for anomaly detection.

[0108] The multi-task anomaly detector 40 communicates with the NeRFXL fusion engine 30 to construct a hierarchical Transformer architecture. A modal gating mechanism is introduced in the self-attention layer to realize collaborative analysis and anomaly detection of multimodal data.

[0109] The dynamic knowledge graph engine 50 is connected to the multi-task anomaly detector 40. It constructs a knowledge graph update module based on graph neural network, defines a relation propagation function, realizes the improvement from data to knowledge, and supports intelligent decision-making and response plan generation.

[0110] Preferably, the present invention further includes a real-time early warning module 60, which is communicatively connected to the dynamic knowledge graph engine 50 and the multi-task anomaly detector 40, for generating early warning signals and pushing out handling plans.

[0111] The technical solutions for each module of the system will be explained in detail below.

[0112] like Figure 2 As shown, the heterogeneous data spatiotemporal alignment module 10 based on 5GMEC includes a two-level clock synchronization unit 11, a coordinate system unit 12, and an objective function optimization unit 13.

[0113] The two-stage clock synchronization unit 11 implements the hardware-level IEEE 1588 precise time protocol and the software-level NTP time synchronization, ensuring microsecond-level (μs) time synchronization accuracy. In a specific embodiment of the present invention, the hardware-level IEEE 1588 protocol is implemented through a dedicated time synchronization chip, ensuring that the clock deviation of each node in the network is <1μs; the software-level NTP time synchronization serves as an auxiliary means to ensure that millisecond-level time consistency can still be maintained even when the hardware-level synchronization fails.

[0114] Unit 12 of the coordinate system is used to establish the global coordinate system transformation tree and define the transformation relationships:

[0115]

[0116] Where: x g x is a coordinate vector in the global coordinate system, a homogeneous coordinate with dimensions 4×1; local This is a coordinate vector in the local coordinate system, also a homogeneous coordinate of 4×1; This is a 4×4 homogeneous transformation matrix, including rotation and translation transformations; · denotes matrix multiplication. This transformation tree supports multi-level coordinate systems, ensuring that data from different sensors can be fused and analyzed within the same spatial reference frame.

[0117] Objective function optimization unit 13 establishes the registration objective function:

[0118]

[0119] Where: E(R,t) is the error function; R is the rotation matrix; t is the translation vector; p i Let q be the coordinates of the i-th point in the source point cloud, which is a 3×1 vector; i The coordinates of the corresponding point in the target point cloud (BIM model) are also represented as a 3×1 vector; N is the total number of matching point pairs, typically selected from feature point pairs such as corner points and edge points; |·| 2 ∑ represents the squared Euclidean distance; ∑ represents the summation operation.

[0120] This invention solves for the optimal transformation parameters through SVD decomposition. The processing flow is as follows: First, the centroids of the source and target point clouds are calculated.

[0121]

[0122] Then calculate the decentralized point set: p′ i =p i -μ p ,q′ i =q i -μ q ;

[0123] Construct the covariance matrix:

[0124] Perform SVD decomposition on the covariance matrix: H = USV T ;

[0125] Calculate the optimal rotation matrix: R = VU T ;

[0126] Finally, calculate the translation vector: t = μ q -Rμp .

[0127] Preferably, the improved Fast-ICP algorithm in this invention significantly improves point cloud registration accuracy by introducing weighting factors and local feature descriptors. Specifically, different weights are assigned to different types of feature points, such as corner points with a weight of 1.0, edge points with a weight of 0.8, and planar points with a weight of 0.6. Simultaneously, local feature descriptors are used to optimize the search for corresponding points, reducing the probability of incorrect matching. After optimization, the registration error is significantly reduced from ±5mm in the traditional ICP algorithm to ±1.5mm, meeting the requirements for precision construction monitoring.

[0128] like Figure 3 As shown, the feature-level fusion network 20 includes a geometric feature extraction unit 21, a temporal feature extraction unit 22, and a feature association unit 23, which realizes the effective fusion of point cloud geometric features and sensor temporal features.

[0129] The geometric feature extraction unit 21 uses an improved PointNet++ architecture to process point cloud data, extracting point cloud features through a multilayer perceptron. The calculation formula is as follows:

[0130] f θ (p i )=σ(W (2) (σ(W (1) p i +b (1) ))+b (2) ),

[0131] Where: f θ (p i Let p be a point. i The feature representation has a dimension of 64; W (1) The first layer weight matrix, with dimensions 3×32, maps 3D point coordinates to 32-dimensional features; W (2) This is the second-layer weight matrix, with dimensions of 32×64, achieving feature dimensionality upscaling; b (1) b is the first-level bias vector, with a dimension of 32×1; (2) The second layer bias vector has a dimension of 64×1; σ is the ReLU activation function, defined as σ(x)=max(0,x).

[0132] In a preferred embodiment of the present invention, the geometric feature branch uses a modified unscented Kalman filter (UKF) to process the point cloud normal vector, and the calculation formula is as follows:

[0133]

[0134] Where: n i p is the normal vector of the i-th point, with dimensions 3×1; iLet be the 3D coordinate vector of the i-th point in the point cloud; × represents the cross product of vectors; |·| represents the magnitude of the vector. For boundary points, the normal vector is calculated using the set of k nearest neighbors to avoid index out-of-bounds errors.

[0135] The advantage of UKF filtering lies in its ability to handle nonlinear systems more accurately without requiring linearization assumptions. The specific implementation steps are as follows: First, a set of sigma points are selected, capturing the mean and covariance of the state distribution; then, these sigma points are propagated through the nonlinear system; finally, a new state estimate and covariance are reconstructed through a weighted average. This method is particularly suitable for handling point cloud noise, effectively filtering out outliers and improving the accuracy of normal vector calculation.

[0136] The temporal feature extraction unit 22 processes sensor data using a bidirectional LSTM and employs a temporal attention mechanism. The calculation formula is as follows:

[0137] h t =σ(W h [h t-1 ,x t ]+b h ),

[0138] α t =softmax(v T tanh(W h h t +W s s)),

[0139] Where: h t h is the hidden state vector at time t, with a dimension of 128; t-1 Let x be the hidden state vector at time t-1; t W is the input vector at time t, containing sensor data such as stress and temperature; h b is a weight matrix used to connect the hidden state from the previous time step with the current input; h σ is the bias vector; σ is the activation function, here we use the tanh function; α t Let be the attention weights at time t; v be the learnable attention vector with the same dimension as the hidden layer; W ... s is the global context transformation matrix; s is the global context vector; tanh is the hyperbolic tangent activation function.

[0140] Feature association unit 23 is used to establish the cross-modal covariance matrix, and the calculation formula is as follows:

[0141] Σ XY =E[(X-μ)] X )(Y-μ Y ) T ],

[0142] Wherein: ΣXY is the covariance matrix; X is the geometric eigenvector, such as the curvature and normal vector of a point cloud; Y is the temporal eigenvector, such as the frequency and amplitude of sensor data; μ X Let μ be the mean vector of X; Y Let Y be the mean vector; E[·] represents the expected value; (·) T This indicates the matrix transpose.

[0143] Canonical correlation analysis (CCA) is used to correlate features and find the maximum correlation between two sets of feature vectors. The specific steps are as follows:

[0144] Calculate the autocovariance matrix Σ XX and Σ YY ;

[0145] Solve the characteristic equation Obtain the projection vector w;

[0146] Finally, the correlation coefficient was calculated.

[0147] Preferably, this invention employs an adaptive threshold when calculating correlation, dynamically adjusting the correlation coefficient threshold based on the data distribution. For example, when the point cloud data has high noise, the threshold is set to 0.7; when the sensor data has a high signal-to-noise ratio, the threshold is increased to 0.85. This adaptive strategy achieves a cross-modal feature association accuracy of 98.7%, significantly better than the 85% of traditional methods.

[0148] like Figure 4 As shown, the NeRFXL fusion engine 30 includes a spatial hash coding unit 31, a dynamic density modeling unit 32, and a real-time rendering unit 33, which realizes efficient fusion and high-quality visualization of point cloud and video data.

[0149] Spatial hash coding unit 31 maps the input coordinates to a multi-resolution hash table, and the calculation formula is as follows:

[0150] γ(x)=[h1(x),...,h L (x)],

[0151] Where: γ(x) is the hash code vector of coordinate x; h L (x) is the L-th level hash function, mapping coordinates to hash table indices; L is the number of hash levels, set to 16 in this invention; [.,...,.] represents vector concatenation operations. The spatial resolution of each level starts from 2... 2 to 2 16 It covers multi-scale spatial representation from centimeter to millimeter.

[0152] The hash function is implemented using a space-consistent hashing algorithm:

[0153]

[0154] in T represents rounding down. l This is the size of the l-th level hash table, usually set to 2. 19 This multi-resolution hash structure can efficiently capture spatial details while maintaining memory usage efficiency.

[0155] The dynamic density modeling element 32 defines a deformable radiation field function, and the calculation formula is as follows:

[0156] σ(x,δ)=MLP(γ(x+Δ(x,δ))),

[0157] Where: σ(x,δ) is the volume density at position x; δ is the deformation parameter, such as the displacement of the support structure; Δ(x,δ) is the deformation field function, which models the deformation of the object; MLP represents a multilayer perceptron, which consists of multiple fully connected layers. The specific structure is a 5-layer network, with the dimensions of each layer being [64,128,256,128,1].

[0158] The deformation field function Δ(x,δ) adopts a physics-based deformation model, considering the effects of material elasticity and gravity: Δ(x,δ)=δ·φ(x)·G(x),

[0159] Where φ(x) is the spatial influence factor and G(x) is the gravity influence factor. When the support structure deforms, the model can automatically adapt and update the radiation field expression by adjusting the δ parameter.

[0160] The radiation field function is defined as:

[0161] C(r)=∫T(t)σ(x(t))c(x(t),d)dt,

[0162] Where: C(r) is the color value of light r; T(t) is the transmittance, representing the probability that the light ray is not blocked from the starting point to position t, calculated by the following formula: σ(x(t)) is the volume density at position x(t); c(x(t),d) is the color value at position x(t) along direction d, which is usually predicted by another MLP network; ∫ represents the integral operation, which is approximated by discrete sampling points in actual implementation.

[0163] The real-time rendering unit 33 supports interactive viewpoint rendering at 4096×2160@30fps, employing a layered sampling strategy and GPU parallel computing to accelerate the rendering process. Specifically, this includes: a coarse sampling stage, where 64 points are uniformly sampled along the light rays; and a fine sampling stage, where, based on the density distribution of the coarse sampling results, an additional 128 points are sampled in high-density areas. This emphasis on importance sampling significantly improves rendering efficiency, and the GPU's parallel computing capabilities enable millisecond-level rendering response times.

[0164] Preferably, the NeRFXL fusion engine of this invention employs a progressive training strategy, starting from a low resolution and gradually increasing to the target resolution, significantly reducing training time. In experiments, compared to the traditional NeRF model, training time was reduced by 70%, while rendering quality and accuracy were significantly improved.

[0165] like Figure 5 As shown, the multi-task anomaly detector 40 includes a modality-specific coding unit 41, a gated cross-attention unit 42, a three-dimensional spatiotemporal attention unit 43, and a hybrid density scoring unit 44, realizing collaborative analysis and anomaly detection of multimodal data.

[0166] Modality-specific coding unit 41 is designed with independent position coding for each mode, and the calculation formula is as follows:

[0167]

[0168] Where: PE is the location code; pos is the spatiotemporal location index, such as the point number or time step in the point cloud; i is the coding dimension index; d is the total dimension of the location code, which is set to 128 in this invention; and sin is the sine function. For point cloud data, the location code captures spatial location information; for sensor data, the location code represents the sequential relationship in the time series.

[0169] Gated cross-attention unit 42 calculates cross-modal attention. The attention calculation formula is as follows:

[0170]

[0171] Where: Q is the query matrix with dimensions n×d k K is the bond matrix with dimensions m×d. k V is a value matrix with dimensions m×d. v ;d k Let G be the dimension of the key vector; G is the modal gating matrix with dimension n×m and element values ​​in the interval [0,1]; ⊙ represents element-wise multiplication; softmax is the normalized exponential function, defined as...

[0172] The modal gating matrix G adaptively adjusts the attention weights between different modalities through learnable parameters: G ij =sigmoid(W g [h i ,k j ]), h i For the query vector, k j W is the key vector. g The parameter matrix is ​​learnable, and sigmoid is a sigmoid activation function that maps the output to the interval [0,1].

[0173] The three-dimensional spatiotemporal attention unit 43 constructs a space-time joint attention mechanism, and the calculation formula is as follows:

[0174]

[0175] Where: H is the spatial dimension, representing the voxel grid size, which is set to 64×64 in this invention; T is the time series length, dividing the sensor time series data into 60-second periods; q i Let k be the query vector for the i-th spatial location; j v is the key vector at the j-th time step; j Let be the value vector at the j-th time step; d is the vector dimension; exp is the exponential function.

[0176] Preferably, this invention implements parallel attention computation with eight heads, each focusing on a different part of the feature space. The results from the multiple heads are then concatenated and subjected to a linear transformation to obtain the final output. This multi-head mechanism can capture richer feature relationships and improve the model's expressive power.

[0177] Mixed density scoring unit 44 calculates the anomaly probability using the following formula:

[0178]

[0179] Where: p(y) is the anomaly probability distribution; K is the Gaussian mixture factor, set to 3 in this invention, corresponding to low, medium, and high risk levels respectively; π k Let be the mixing coefficient of the k-th Gaussian distribution, satisfying μ k Let Σ be the mean vector of the k-th distribution; k Let N(·|·,·) be the covariance matrix of the k-th distribution; N(·|·,·) represents the probability density function of the multivariate Gaussian distribution, defined as...

[0180] Mixing coefficient π k Mean vector μ k The covariance matrix Σ k The model is optimized iteratively using the Expectation-Maximization (EM) algorithm. The initialization phase utilizes K-means clustering results, and then the model parameters are continuously optimized through alternating iterations of the EM algorithm until convergence or the maximum number of iterations is reached.

[0181] Preferably, a composite anomaly alarm is triggered when a local curvature κ > 0.15 and a frequency change Δf > 8 Hz are detected. These thresholds are determined based on extensive engineering experience. A curvature threshold of 0.15 corresponds to significant deformation of the support structure, and a frequency change of 8 Hz indicates a significant change in the structure's dynamic characteristics. The simultaneous occurrence of both indicates a serious safety hazard.

[0182] like Figure 6 As shown, the dynamic knowledge graph engine 50 includes a secure ontology definition unit 51, a graph update unit 52, a heterogeneous graph attention network unit 53, and a multi-hop reasoning unit 54, which realizes the improvement from data to knowledge and supports intelligent decision-making.

[0183] Safety ontology definition unit 51 constructs the construction safety ontology:

[0184]

[0185] Where: O is the ontology; ε is the set of entities, including entity types such as equipment, materials, processes, and environment; A is a set of relationships, including causal, composition, and dependency relationships; A is a set of axiomatic constraints, defining the logical rules between entities. In this invention, the safety ontology contains 500+ entity types and 80+ relationship types, covering the main knowledge in the field of construction safety.

[0186] The spectrum update unit 52 implements event-driven spectrum updates, triggering a spectrum update when the detected stress value meets the following conditions:

[0187] σ t >μ+3δ,

[0188] Where: σ t Let t be the stress value detected at time t; μ be the mean of historical stress data; and δ be the standard deviation of historical stress data. This condition is based on the 3σ principle of normal distribution, and theoretically, the probability of anomalies is less than 0.3%, which can effectively filter out random fluctuations and capture true anomalies.

[0189] Heterogeneous graph attention network unit 53 defines the node update formula:

[0190]

[0191] in: Let K be the feature representation of node v in layer l+1; K is the number of attention heads, which is set to 8 in this invention. W represents the attention weight of the node pair (u,v) in the k-th attention head; k Let be the weight matrix for the k-th attention head; Let be the feature representation of node u at layer l; N(v) be the set of neighboring nodes of node v; σ is the ELU activation function, defined as σ(x) = x if x>0 else α(e x -1), where α is usually set to 1; || represents the multi-head attention splicing operation.

[0192] Attention weight α u v k The calculation is as follows:

[0193]

[0194] in a k Let be a learnable attention vector, and || denotes vector concatenation.

[0195] Multi-hop reasoning unit 54 calculates the probability of relationships using the following formula:

[0196] P(h|r,t)=σ(h T ·diag(r)·t),

[0197] Where: P(h|r,t) is the probability that a relation r exists between a given head entity h and a tail entity t; diag(r) is the matrix formed by diagonalizing the relation vector r; σ is the sigmoid activation function, which maps the output to the interval [0,1]; (·) T This indicates the transpose operation.

[0198] Multi-hop reasoning supports complex path queries, such as determining what measures to take if the support structure displacement is too large. Through relationship combination and path search, indirectly related entities and relationships can be discovered, providing more comprehensive decision support. In this invention, emergency response plans can be quickly generated by calculating the probability of emergency response paths within a 3-hop range.

[0199] Preferably, the dynamic knowledge graph of the present invention employs an incremental learning strategy, continuously adjusting and optimizing the graph structure and parameters based on newly observed data, thus avoiding the static nature and update lag issues of traditional knowledge graphs. In practical applications, the knowledge graph reasoning response time is less than 500ms, which can meet the needs of real-time decision-making.

[0200] As previously mentioned, the present invention also includes a real-time early warning module 60, which is communicatively connected to the dynamic knowledge graph engine 50 and the multi-task anomaly detector 40. This module has two main functions:

[0201] 1. When the 3D heatmap generated by the NeRF engine shows a displacement value exceeding a threshold, a yellow warning signal is generated. Specifically, differentiated warning thresholds are set according to different structural and material characteristics, such as a displacement threshold of 10mm for steel structure support and 5mm for concrete support. Warning levels are divided into three categories: green (safe), yellow (warning), and red (danger), visually displaying the safety status.

[0202] 2. Disposal plans, including stopping excavation, adding supports, and initiating dewatering, are pushed to terminal devices via the 5G network. These plans are automatically generated based on the type and severity of the anomaly, taking into account contextual information such as the construction environment and geological conditions to ensure their relevance and feasibility.

[0203] Preferably, the real-time early warning module also possesses predictive maintenance capabilities, capable of providing an early warning of potential risks exceeding limits 10 minutes in advance. This capability is based on trend analysis, predicting future conditions by fitting the changing trends of historical data. For example, when a change in the misalignment rate is detected... In this case, the system can predict when the safety threshold will be exceeded under the condition of continuous change rate, and issue an early warning.

[0204] The workflow and effects of this invention are illustrated below through two specific application examples.

[0205] like Figure 7 As shown, the workflow of this invention applied to real-time monitoring of foundation pit construction is as follows:

[0206] 1. The edge node receives point cloud data from a laser scanner (2,000,000 points / second) and data from 10 stress sensors;

[0207] 2. The spatiotemporal alignment module registers the BIM model with the real-time point cloud, with a registration error of <2mm;

[0208] 3. The fusion network detected a local curvature abrupt change region (κ>0.15), and the synchronization sensor showed an abnormal frequency (Δf>8Hz);

[0209] 4. The NeRF engine generates a 3D heat map, showing that the displacement of the support structure has reached the warning value (yellow warning);

[0210] 5. Knowledge graph automatically pushes disposal solutions: Stop excavation → Add support → Start dewatering.

[0211] In this application example, the system completes the analysis and pushes out the treatment plan within 10 seconds after detecting the abnormality of the support structure. This is several hours earlier than traditional manual inspection, effectively preventing the expansion of foundation pit deformation and potential safety accidents.

[0212] like Figure 8 As shown, the workflow of this invention applied to the quality inspection of shield tunnel segment assembly is as follows:

[0213] 1. Multi-view industrial camera captures images of segment joints (resolution 4096×2160);

[0214] 2. The point cloud fusion module reconstructs the 3D model of the joint and detects the misalignment δ = 4.3 mm (standard value ≤ 5 mm);

[0215] 3. The LSTM module analyzes the hydraulic jack pressure time series data to predict the offset trend.

[0216] 4. The system provides a 10-minute advance warning of potential risks exceeding limits, guiding adjustments to tunneling parameters.

[0217] In this application, the system uses predictive analysis to detect potential risks of substandard assembly quality in advance, giving on-site personnel ample time to adjust parameters, ensuring that the segment assembly quality meets the specifications, and avoiding rework and quality defects.

[0218] The intelligent construction safety monitoring system and method based on multimodal data fusion of the present invention has significant technical advantages:

[0219] 1. Improved accuracy of multi-source data fusion: Through the improved Fast-ICP algorithm and dual-stream depthwise separable convolutional network, the point cloud registration error is controlled within ±1.5mm, and the cross-modal feature association accuracy reaches 98.7%, which is far superior to traditional methods.

[0220] 2. Improved anomaly detection efficiency: Based on the hierarchical Transformer architecture and modal gating mechanism, the 3D defect recognition speed reaches 120 frames / second, and the multimodal anomaly detection F1-score reaches 0.927, significantly reducing the false alarm rate.

[0221] 3. System response performance optimization: Based on the 5GMEC edge computing architecture, the end-to-end latency is controlled within 200ms, and the knowledge graph inference response time is less than 500ms, meeting the needs of real-time monitoring and decision-making.

[0222] 4. Predictive maintenance capability: Through trend analysis and dynamic knowledge graph, it can provide early warning of potential risks exceeding the standard 10 minutes in advance and automatically generate targeted handling plans, shifting from passive response to proactive prevention.

[0223] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal data fusion-based intelligent construction safety monitoring system, characterized in that, include: The heterogeneous data spatiotemporal alignment module based on 5GMEC is used to realize the coordinate system transformation between laser point cloud and BIM model through the improved Fast-ICP algorithm, and to establish a rigid transformation matrix T=[R|t], where: R is the rotation matrix, satisfying R∈SO(3), SO(3) is a three-dimensional special orthogonal group; t is the translation vector, satisfying t∈R 3 R 3 Represents the three-dimensional real number space; A feature-level fusion network, communicatively connected to the heterogeneous data spatiotemporal alignment module, is used to construct a two-stream depthwise separable convolutional network, including a geometric feature branch and a temporal feature branch. The geometric feature branch uses an improved unscented Kalman filter to process the point cloud normal vectors, and the calculation formula is as follows: Where: n i p is the normal vector of the i-th point; i Let be the 3D coordinate vector of the i-th point in the point cloud; × represents the cross product of vectors; |·| represents the magnitude of the vector. The time-series feature branch processes stress sensor data using LSTM, and the calculation formula is as follows: h t =σ(W h [h t-1 ,x t ]+b h ), Where: h t h is the hidden state vector at time t; t-1 Let x be the hidden state vector at time t-1; t W is the input vector at time t; h b is the weight matrix; h σ is the bias vector; σ is the activation function; [·,·] denotes vector concatenation operation; The NeRFXL fusion engine, which communicates with the feature-level fusion network, is used to fuse point cloud and video data using multi-scale neural radiation fields, and to define the radiation field function: C(r)=∫T(t)σ(x(t))c(x(t),d)dt, Where: C(r) is the color value of light r; T(t) is the transmittance; σ(x(t)) is the volume density at position x(t); c(x(t),d) is the color value at position x(t) along direction d; ∫ represents the integral operation; A multi-task anomaly detector, communicating with the NeRFXL fusion engine, is used to construct a hierarchical Transformer architecture. A modal gating mechanism is introduced in the self-attention layer, and the attention calculation formula is as follows: Where: Q is the query matrix; K is the key matrix; V is the value matrix; d k is the dimension of the key vector; G is the modal gating matrix with elements in the interval [0,1]; ⊙ represents element-wise multiplication; softmax is the normalized exponential function; The dynamic knowledge graph engine, which communicates with the multi-task anomaly detector, is used to construct a knowledge graph update module based on a graph neural network and defines a relation propagation function. in: Here is the feature representation of node v at layer l+1; W represents the feature representation of node u at layer l. (l) b is the weight matrix of the l-th layer; (l) σ is the bias vector of the l-th layer; N(v) is the set of neighboring nodes of node v; σ is the activation function; ∑ represents the summation operation.

2. The system according to claim 1, characterized in that, The heterogeneous data spatiotemporal alignment module includes: A two-stage clock synchronization unit is used to implement the hardware-level IEEE 1588 precise time protocol and the software-level NTP time synchronization. The coordinate system unit is used to establish the global coordinate system transformation tree and define the transformation relationships: Where: x g x is the coordinate vector in the global coordinate system; local It is a coordinate vector in the local coordinate system; A 4×4 homogeneous transformation matrix containing rotation and translation transformations; · denotes matrix multiplication; the objective function optimization unit is used to establish the registration objective function: Where: E(R,t) is the error function; R is the rotation matrix; t is the translation vector; p i Let q be the coordinates of the i-th point in the source point cloud; i The coordinates of the corresponding point in the target point cloud; N is the total number of matching point pairs; |·| 2 ∑ represents the squared Euclidean distance; ∑ represents the summation operation.

3. The system according to claim 1, characterized in that, The feature-level fusion network includes: The geometric feature extraction unit processes point cloud data using an improved PointNet++ architecture, extracting point cloud features through a multilayer perceptron. The calculation formula is as follows: f θ (p i )=σ(W (2) (σ(W (1) p i +b (1) ))+b (2) ), Where: f θ (p i Let p be a point. i Feature representation; W (1) W is the first layer weight matrix; (2) b is the weight matrix for the second layer; (1) b is the first layer bias vector; (2) σ is the second-layer bias vector; σ is the activation function. The temporal feature extraction unit processes sensor data using a bidirectional LSTM and employs a temporal attention mechanism. The calculation formula is as follows: α t =softmax(v T tanh(W h h t +W s s)), Where: α t Let be the attention weight at time t; v be the learnable attention vector; W ... h h is the hidden state transformation matrix; t Let W be the hidden state at time t; s s is the global context transformation matrix; s is the global context vector; tanh is the hyperbolic tangent activation function; the feature association unit is used to establish the cross-modal covariance matrix, and its calculation formula is: S XY =E[(X-μ X )(Y-μ Y ) T ], Wherein: Σ XY X is the covariance matrix; Y is the geometric eigenvector; μ is the temporal eigenvector; X is the geometric eigenvector; μ is the temporal eigenvector; μ is the ... X Let μ be the mean vector of X; Y Let Y be the mean vector; E[·] represents the expected value; (·) T This indicates the matrix transpose.

4. The system according to claim 1, characterized in that, The NeRFXL fusion engine includes: Spatial hash coding unit, used to map input coordinates to a multi-resolution hash table, is calculated using the following formula: γ(x)=[h1(x),...,h L (x)], Where: γ(x) is the hash code vector of coordinate x; h L (x) is the L-th level hash function, which maps coordinates to hash table indices; L is the hash level; [·,…,·] represents vector concatenation operation; Dynamic density modeling elements are used to define deformable radiation field functions, and the calculation formula is as follows: σ(x,δ)=MLP(γ(x+Δ(x,δ))), Where: σ(x,δ) is the volume density at position x; δ is the deformation parameter, such as the displacement of the support structure; Δ(x,δ) is the deformation field function; MLP represents a multilayer perceptron, which consists of multiple fully connected layers; Real-time rendering unit, used to support interactive viewpoint rendering.

5. The system according to claim 1, characterized in that, The multi-task anomaly detector includes: Modality-specific coding units (MSCs) are used to design independent position codes for each mode, and their calculation formula is as follows: PE(pos,2i)=sin(pos / 10000 2i / d ), Where: PE is the location code; pos is the spatiotemporal location index; i is the coding dimension index; d is the total dimension of the location code; sin is the sine function; A gated cross-attention unit is used to compute cross-modal attention, as described in claim 1; A three-dimensional spatiotemporal attention unit is used to construct a joint spatial-temporal attention mechanism. The calculation formula is as follows: Where: H is the spatial dimension, representing the voxel grid size; T is the time series length; q i Let k be the query vector for the i-th spatial location; j v is the key vector at the j-th time step; j Let be the value vector at the j-th time step; d is the vector dimension; exp is the exponential function; The mixed density scoring unit is used to calculate the probability of anomalies, and the calculation formula is as follows: Where: p(y) is the outlier probability distribution; K is the Gaussian mixture component number; π k μ is the mixing coefficient of the k-th Gaussian distribution; k Let Σ be the mean vector of the k-th distribution; k Let N(·|·,·) be the covariance matrix of the k-th distribution; N(·|·,·) represents the probability density function of the multivariate Gaussian distribution.

6. The system according to claim 1, characterized in that, The dynamic knowledge graph engine includes: Safety ontology definition unit, used to construct the construction safety ontology: Where: O is the ontology; ε is the set of entities; Let A be the set of relations; and let A be the set of axiom constraints. The spectrum update unit is used to implement event-driven spectrum updates. The spectrum update is triggered when the detected stress value meets the following conditions: s t >μ+3d, Where: σ t denoted as t, where μ is the stress value detected at time t; μ is the mean of historical stress data; and δ is the standard deviation of historical stress data. Heterogeneous graph attention network unit, used to define node update formula; in: Let K be the feature representation of node v at layer l+1; K is the number of attention heads. W represents the attention weight of the node pair (u,v) in the k-th attention head; k Let be the weight matrix for the k-th attention head; Let be the feature representation of node u at layer l; N(v) be the set of neighboring nodes of node v; σ be the activation function; || denotes the multi-head attention concatenation operation; The multi-hop inference unit is used to calculate the probability of a relationship; the calculation formula is as follows: P(h|r,t)=σ(h T ·diag(r)·t), Where: P(h|r,t) is the probability that a relation r exists between a given head entity h and a tail entity t; diag(r) is the matrix formed by diagonalizing the relation vector r; σ is the sigmoid activation function, which maps the output to the interval [0,1]; (·) T This indicates the transpose operation.

7. The system according to claim 4, characterized in that, The spatial hash coding unit divides the three-dimensional space into 16 resolution levels, with each level's spatial resolution ranging from 2... 2 to 2 16 .

8. The system according to claim 1, characterized in that, It also includes a real-time early warning module, which is communicatively connected to the dynamic knowledge graph engine and the multi-task anomaly detector, and is used for: When the displacement value displayed in the 3D heatmap generated by the NeRF engine exceeds the threshold, a yellow warning signal is generated; The 5G network will push disposal plans to terminal devices, including stopping excavation, adding support, and starting dewatering.

9. The system according to claim 5, characterized in that, The three-dimensional spatiotemporal attention unit divides the point cloud data into a 64×64 grid partition and divides the sensor time-series data into 60-second cycles to achieve parallel attention calculation for 8 heads.

10. A method for intelligent construction safety monitoring based on multimodal data fusion, characterized in that, include: The spatiotemporal registration of laser point clouds and BIM models is achieved using an improved Fast-ICP algorithm. The rigid transformation matrix T = [R|t] is solved, where: R is the rotation matrix, satisfying R∈SO(3), SO(3) is a three-dimensional special orthogonal group; t is the translation vector, satisfying t∈R 3 R 3 Represents the three-dimensional real number space; A dual-stream depthwise separable convolutional network is used to simultaneously process point cloud normal vectors and stress sensor temporal data. The formula for calculating point cloud normal vectors is as follows: Where: n i p is the normal vector of the i-th point; i Let be the 3D coordinate vector of the i-th point in the point cloud; × represents the cross product of vectors; |·| represents the magnitude of the vector. A real-time 3D heatmap with a resolution of 4096×2160 was generated using a multi-scale neural radiation field fusion engine. The radiation field function is: C(r)=∫T(t)σ(x(t))c(x(t),d)dt, Where: C(r) is the color value of light r; T(t) is the transmittance; σ(x(t)) is the volume density at position x(t); c(x(t),d) is the color value at position x(t) along direction d; ∫ represents the integral operation; Cross-modal attention weights are calculated using a hierarchical Transformer architecture. The attention calculation formula is as follows: Where: Q is the query matrix; K is the key matrix; V is the value matrix; d k is the dimension of the key vector; G is the modal gating matrix with elements in the interval [0,1]; ⊙ represents element-wise multiplication; softmax is the normalized exponential function; The node update formula for dynamically updating the construction safety knowledge graph based on a graph neural network is as follows: in: Here is the feature representation of node v at layer l+1; W represents the feature representation of node u at layer l. (l) b is the weight matrix of the l-th layer; (l) σ is the bias vector of the l-th layer; N(v) is the set of neighboring nodes of node v; σ is the activation function; ∑ represents the summation operation.

Images