Electromechanical installation engineering safety management early warning method and system based on large model

Abstract

The invention discloses an electromechanical installation engineering safety management early warning method and system based on a large model. The method comprises the following steps: acquiring multi-modal data; performing space-time synchronization and feature level fusion on the multi-modal data by using a cross-modal alignment algorithm to obtain fused multi-modal feature data; constructing dynamic space-time diagram structure data based on the fused multi-modal feature data, inputting the dynamic space-time diagram structure data into an STGCN time sequence diagram convolutional network, distinguishing high-risk behavior features through a diagram attention mechanism, and outputting space-time feature data; and inputting the spatial-temporal characteristic data into an MANN memory enhanced neural network, outputting an installation engineering risk index, performing evaluation according to the installation engineering risk index, and triggering hierarchical early warning to generate an intervention strategy. The safety management efficiency is improved, the labor cost is reduced, and the safety accident rate is reduced.

Description

Technical Field

[0001] This invention relates to the field of building engineering safety management technology, and in particular to a method and system for early warning of safety management in electromechanical installation engineering based on a large model. Background Technology

[0002] Electromechanical installation projects involve complex construction sites with high personnel turnover and diverse equipment. High-altitude operations, equipment hoisting, and electrical operations all pose significant safety risks, making safety management extremely challenging. Current traditional safety management models rely primarily on manual inspections and paper records, which suffer from significant drawbacks such as delayed response, low accuracy, and limited coverage, making them ill-suited to addressing multi-scenario and dynamic safety hazards. Existing early warning technologies often depend on single-type data, enabling only localized hazard identification. They lack the integrated utilization of multi-dimensional data, including visual, spatial, equipment operational, and management standards data, leading to missed or false diagnoses and hindering early risk prediction and tiered risk control. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing a safety management and early warning method and system for electromechanical installation engineering based on a large model.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following steps in the above-mentioned early warning method for safety management of electromechanical installation engineering based on a large model: Acquire visual data, point cloud data, sensor data, and management document data from the construction site of electromechanical installation engineering, and preprocess the collected data to obtain multimodal data; The multimodal data is spatiotemporally synchronized and feature-level fused using a cross-modal alignment algorithm to obtain fused multimodal feature data; Based on the fusion of multimodal feature data, a dynamic spatiotemporal graph structure is constructed and input into the STGCN temporal graph convolutional network. High-risk behavioral features are distinguished through the graph attention mechanism, and spatiotemporal feature data is output. The spatiotemporal feature data is input into the MANN memory-enhanced neural network, which outputs an installation engineering risk index. The risk index is used for evaluation, and a graded early warning system is triggered to generate an intervention strategy.

[0005] Furthermore, in the aforementioned early warning method for safety management of electromechanical installation engineering based on a large model, the acquisition of visual data, point cloud data, sensor data, and management document data from the construction site of the electromechanical installation engineering, and the preprocessing of the collected data to obtain multimodal data, including: Acquire visual data, point cloud data, sensor data, and management document data from the construction site of electromechanical installation projects; Image enhancement algorithms are used to eliminate noise caused by lighting changes and dust occlusion in visual data. Frames are extracted from video data to retain key action frames. Human and equipment targets are located through target detection algorithms. After cropping invalid areas, the image size is unified. By using a pass-through filtering algorithm to remove environmental interference points and equipment scanning noise points from the point cloud data, the processed point cloud data is downsampled, and the point cloud data is uniformly converted into the global coordinate system of the construction site through coordinate calibration. Interpolation is used to fill in missing data from the sensor data, Kalman filtering is used to eliminate errors caused by sensor jitter, and the values ​​of different types of sensors are standardized. Based on NLP (Natural Language Processing) technology, unstructured text in management document data is segmented and stop word removed to extract information and convert it into structured data, resulting in multimodal data.

[0006] Furthermore, in the aforementioned early warning method for safety management of electromechanical installation engineering based on a large model, the step of using a cross-modal alignment algorithm to perform spatiotemporal synchronization and feature-level fusion of the multimodal data to obtain fused multimodal feature data includes: Based on the timestamps of sensor data, the frame times of visual data, the scanning times of point cloud data, and the update times of management document data are aligned to obtain time-aligned data. Using the global coordinate system of point cloud data in the time-aligned data as a reference, the pixel coordinates of the visual data are converted into spatial coordinates through camera calibration parameters, and accurately aligned with the spatial position of the point cloud data. At the same time, combined with the construction site zoning information in the management document data, various types of data are mapped to the corresponding construction areas to obtain spatiotemporally aligned data.

[0007] Furthermore, in the aforementioned early warning method for safety management of electromechanical installation engineering based on a large model, the step of using a cross-modal alignment algorithm to perform spatiotemporal synchronization and feature-level fusion of the multimodal data to obtain fused multimodal feature data includes: Human motion features, device appearance features, and environmental scene features are extracted from spatiotemporally aligned data using a CNN convolutional neural network. Spatial structure features, equipment installation accuracy features, and workspace spacing features were extracted from the spatiotemporal alignment data using the PointNet++ algorithm. The device operating status characteristics, personnel physiological characteristics, and environmental anomaly characteristics in the spatiotemporally aligned data are extracted using an MLP (Multilayer Perceptron). Safety rule features, risk threshold features, and personnel qualification features are extracted from spatiotemporally aligned data using the BERT large model. An attention-based cross-modal fusion algorithm is used to perform weighted fusion of various modal features. Weights are assigned according to the importance of each modal data under different construction scenarios to obtain fused multimodal feature data.

[0008] Furthermore, in the aforementioned early warning method for safety management of electromechanical installation engineering based on a large model, the dynamic spatiotemporal graph structure data constructed based on the fusion of multimodal feature data is input into the STGCN temporal graph convolutional network. High-risk behavioral features are distinguished through a graph attention mechanism, and spatiotemporal feature data is output, including: Multimodal feature data is input into the STGCN temporal graph convolutional network, and a dynamic spatiotemporal graph is constructed with the human joint coordinates of construction workers as the core nodes. Taking each worker on the construction site as an independent individual, the three-dimensional spatial coordinates of 18 key joints of the human body are extracted from visual data and point cloud data. Each joint coordinate is treated as an independent node, and corresponding feature information is added to each node to define it as a node. Two types of edges are constructed: physical connection edges, which connect adjacent joint nodes of the same person based on human physiological structure, representing the physical correlation of human actions; and spatiotemporal connection edges, which connect the same joint nodes of different people.

[0009] Furthermore, in the aforementioned early warning method for safety management of electromechanical installation engineering based on a large model, the dynamic spatiotemporal graph structure data constructed based on the fusion of multimodal feature data is input into the STGCN temporal graph convolutional network. High-risk behavioral features are distinguished through a graph attention mechanism, and spatiotemporal feature data is output, including: STGCN's graph attention mechanism calculates the attention weight of each node, assigning high attention weights to nodes and edges associated with high-risk behaviors. The temporal convolution module of STGCN is used to extract the temporal variation features of nodes and edges, and the graph convolution module is used to extract spatial correlation features. The temporal features and spatial features are then fused to output spatiotemporal feature data.

[0010] Furthermore, in the aforementioned early warning method for safety management of electromechanical installation engineering based on a large model, the step of inputting the spatiotemporal feature data into the MANN memory-enhanced neural network, outputting an installation engineering risk index, evaluating based on the installation engineering risk index, and triggering a graded early warning generation intervention strategy includes: Spatiotemporal feature data are input into the MANN memory-enhanced neural network. The memory module is used to store historical construction risk data, construction site safety standard thresholds, and risk assessment standard information for electromechanical installation projects. The calculation module compares the spatiotemporal feature data with the historical data in the memory module. Through similarity calculation, it matches the most similar historical risk cases, extracts the corresponding risk weights, and combines them with the high-risk indicators in the current spatiotemporal features to calculate the installation project risk index using a weighted summation method.

[0011] Furthermore, in the electromechanical installation engineering safety management early warning system based on a large model, the electromechanical installation engineering safety management early warning system includes the following modules: The multimodal data acquisition module is used to acquire visual data, point cloud data, sensor data, and management document data from the construction site of electromechanical installation engineering, and to preprocess the acquired data to obtain multimodal data. The feature data fusion module is used to perform spatiotemporal synchronization and feature-level fusion of the multimodal data using a cross-modal alignment algorithm to obtain fused multimodal feature data; The spatiotemporal feature extraction module is used to construct a dynamic spatiotemporal graph structure based on fused multimodal feature data. The data is input into the STGCN temporal graph convolutional network, and high-risk behavioral features are distinguished through the graph attention mechanism to output spatiotemporal feature data. The engineering safety management module is used to input the spatiotemporal feature data into the MANN memory augmentation neural network, output the installation engineering risk index, evaluate the installation engineering risk index, and trigger graded early warning to generate intervention strategies.

[0012] Furthermore, in the electromechanical installation engineering safety management and early warning system based on a large model, the spatiotemporal feature extraction module includes the following sub-modules: The computation submodule is used to calculate the attention weight of each node through STGCN's graph attention mechanism, and assign high attention weights to nodes and edges related to high-risk behaviors. The extraction submodule is used to extract the temporal variation features of nodes and edges through the temporal convolution module of STGCN, extract spatial correlation features through the graph convolution module, fuse the temporal features and spatial features, and output spatiotemporal feature data.

[0013] Furthermore, in the electromechanical installation engineering safety management and early warning system based on a large model, the engineering safety management module includes the following sub-modules: The input submodule is used to input spatiotemporal feature data into the MANN memory-enhanced neural network. The memory module is used to store historical construction risk data of electromechanical installation projects, construction site safety specification thresholds, and risk assessment standard information. The evaluation submodule is used by the calculation module to compare the spatiotemporal feature data with the historical data in the memory module. Through similarity calculation, it matches the most similar historical risk cases, extracts the corresponding risk weights, and combines them with the high-risk indicators in the current spatiotemporal features to calculate the installation project risk index using a weighted summation method.

[0014] Its beneficial effects lie in breaking down data barriers by collecting four types of data—visual, point cloud, sensor, and management document data—and performing standardized preprocessing. This comprehensively covers safety information across all dimensions of the construction scenario, solving the problem of missed detections caused by the limited data in traditional methods. It improves the accuracy of risk identification by utilizing cross-modal alignment algorithms to achieve spatiotemporal synchronization and feature fusion of multiple data sources. Combined with the STGCN temporal graph convolutional network to capture spatiotemporal correlation features, and a graph attention mechanism to accurately distinguish high-risk behaviors, effectively reducing the probability of misjudgments and missed detections. It enables early risk prediction and tiered control by using the MANN memory-enhanced neural network combined with historical data to calculate a risk index, classifying risks into four levels and triggering corresponding warnings. It forms a closed-loop management system, automatically generating targeted intervention strategies and synchronously updating management documents, improving safety management efficiency, reducing labor costs, lowering the accident rate, and ensuring the personal and property safety of construction personnel. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0016] Figure 1 This is a schematic diagram of the first embodiment of the electromechanical installation engineering safety management early warning method based on a large model in this invention. Figure 2 This is a schematic diagram of the second embodiment of the electromechanical installation engineering safety management early warning method based on a large model in this invention. Figure 3 This is a schematic diagram of the first embodiment of the electromechanical installation engineering safety management and early warning system based on a large model in this invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0019] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 As shown, a safety management early warning method for electromechanical installation projects based on a large model is proposed. This method includes the following steps: Step 101: Acquire visual data, point cloud data, sensor data, and management document data from the construction site of the electromechanical installation project. Preprocess the collected data to obtain multimodal data. Specifically, in this embodiment, visual data, point cloud data, sensor data, and management document data of the electromechanical installation construction site are acquired; Image enhancement algorithms are used to eliminate noise caused by lighting changes and dust occlusion in visual data. Frame extraction is performed on video data to retain key action frames. Human and equipment targets are located using target detection algorithms, and invalid areas are cropped to unify image size. Environmental interference points and equipment scanning noise points in point cloud data are removed using a pass-through filtering algorithm. The processed point cloud data is downsampled and converted to a unified global coordinate system of the construction site through coordinate calibration. Interpolation methods are used to fill in missing data from sensor data, and Kalman filtering algorithms are used to eliminate errors caused by sensor jitter. Values ​​from different types of sensors are standardized. Based on NLP (Natural Language Processing) technology, unstructured text in management document data is segmented and stop word removed to extract information and convert it into structured data, resulting in multimodal data.

[0020] To address the complexity of electromechanical installation construction sites, four types of core data are comprehensively collected to ensure coverage of the entire process, including environment, equipment, personnel, and management. First, visual data: High-definition cameras and panoramic cameras deployed at key locations on the construction site, such as high-altitude work areas, equipment installation areas, and material storage areas, collect real-time image and video data on worker actions, equipment operating status, and changes in the site environment. The acquisition frequency is set to 10-15 frames per second to ensure clear capture of critical information such as human joint movements and subtle equipment anomalies. Second, point cloud data: LiDAR equipment scans the construction site to collect three-dimensional spatial coordinate data of the construction area, focusing on capturing spatial information such as equipment installation accuracy, workspace spacing, and the location of high-altitude work platforms. The resolution is controlled within 5cm to ensure the accuracy of the spatial data. Third, sensor data. In construction equipment such as cranes, welding machines, distribution boxes, and personal protective equipment for workers such as safety helmets and safety belts, as well as in the construction site environment, various sensors are deployed to collect real-time data on equipment operating parameters such as voltage, current, and speed; personnel physiological status such as heart rate; and environmental parameters such as temperature, humidity, and dust concentration. The sampling frequency is adjusted to 1-10Hz according to the parameter type to ensure the real-time nature of the data. Fourth, document data is managed by collecting text and tabular data such as construction plans, safety specifications, work briefing records, personnel qualification documents, and equipment maintenance records for electromechanical installation projects. Core information such as safety thresholds, risk assessment standards, and personnel work permissions are sorted out to form a structured document dataset.

[0021] The four types of raw data collected were processed in a targeted manner to eliminate redundancy, correct biases, and generate standardized multimodal data. For visual data, image enhancement algorithms were employed, such as histogram equalization and Gaussian denoising to eliminate noise caused by lighting variations and dust occlusion. Frame extraction was performed on video data, retaining key action frames. Simultaneously, target detection algorithms were used to initially locate core targets such as humans and equipment, cropping invalid areas and standardizing the image size to 1080×1920 pixels. For point cloud data, pass-through filtering and Statistical Outlier were used. Removal, a statistical outlier removal algorithm, eliminates environmental interference points and equipment scanning noise points. It downsamples the processed point cloud data to reduce data volume and simultaneously converts the point cloud data to a unified global coordinate system for the construction site through coordinate calibration, ensuring spatial coordinate consistency. For sensor data, interpolation is used to fill in missing data, and Kalman filtering is used to eliminate errors caused by sensor jitter. Values ​​from different types of sensors are standardized, mapping the data to the [0,1] interval to unify the data volume. For management document data, Natural Language Processing (NLP) techniques are used to segment and remove stop words from unstructured text, extracting key information and converting it into structured data. Threshold standards and risk level classifications from safety regulations are organized into a callable rule base. Finally, the four types of preprocessed standardized data are integrated into a multimodal dataset containing visual, point cloud, sensor, and text data.

[0022] Step 102: Use the cross-modal alignment algorithm to perform spatiotemporal synchronization and feature-level fusion of multimodal data to obtain fused multimodal feature data; Specifically, in this embodiment, the timestamp of the sensor data is used as a reference to align the frame time of the visual data, the scanning time of the point cloud data, and the update time of the management document data to obtain time-aligned data. Using the global coordinate system of the point cloud data in the time-aligned data as a reference, the pixel coordinates of the visual data are converted into spatial coordinates through camera calibration parameters and accurately aligned with the spatial position of the point cloud data. At the same time, combined with the construction site zoning information in the management document data, various types of data are mapped to the corresponding construction areas to obtain spatiotemporal aligned data.

[0023] Human motion features, equipment appearance features, and environmental scene features are extracted from the spatiotemporally aligned data using a CNN convolutional neural network. Spatial structure features, equipment installation accuracy features, and work space spacing features are extracted from the spatiotemporally aligned data using the PointNet++ algorithm. Equipment operating status features, personnel physiological features, and environmental anomaly features are extracted from the spatiotemporally aligned data using an MLP multilayer perceptron. Safety rule features, risk threshold features, and personnel qualification features are extracted from the spatiotemporally aligned data using a BERT large model. An attention mechanism-based cross-modal fusion algorithm is used to weight and fuse various modal features, assigning weights according to the importance of each modal data under different construction scenarios to obtain fused multimodal feature data.

[0024] A cross-modal alignment algorithm is employed to perform dual calibration of multimodal data, including temporal and spatial synchronization. For temporal synchronization, the timestamps of sensor data are used as a benchmark to align the frame times of visual data, the scanning times of point cloud data, and the update times of management document data. Through time interpolation and frame matching, this ensures accurate correspondence of various data at the same time point, eliminating time deviations and controlling them within 50ms, achieving precise matching of multi-dimensional data at the same moment. For spatial synchronization, the global coordinate system of point cloud data is used as a benchmark. The pixel coordinates of visual data are converted into spatial coordinates through camera calibration parameters, achieving precise alignment with the spatial location of the point cloud data. Simultaneously, combined with construction site zoning information in the management document data, various data are mapped to corresponding construction areas, such as high-altitude work areas and equipment installation areas, achieving spatial dimension unification and ensuring that different modal data can accurately describe the same construction scene, the same operational behavior, or the same equipment status.

[0025] Based on spatiotemporal synchronization, core features of various modal data are extracted, and feature-level fusion is achieved through a fusion algorithm to obtain fused multimodal feature data. Features of each modality are extracted separately: visual data is processed using a convolutional neural network (CNN) to extract human action features, equipment appearance features, and environmental scene features; point cloud data is processed using the PointNet++ algorithm to extract spatial structure features, equipment installation accuracy features, and workspace spacing features; sensor data is processed using a multilayer perceptron (MLP) to extract equipment operating status features, personnel physiological features, and environmental anomaly features; and text data is processed using the BERT large-scale model to extract safety rule features, risk threshold features, and personnel qualification features. Subsequently, an attention-based cross-modal fusion algorithm is used to weight and fuse various modal features, assigning weights according to the importance of each modality in different construction scenarios, highlighting high-risk related features, and suppressing redundant features. Finally, fused multimodal feature data that can comprehensively and accurately describe the safety status of construction scenarios is generated, providing high-quality input for subsequent risk feature extraction.

[0026] Step 103: Construct a dynamic spatiotemporal graph structure based on fused multimodal feature data. Input the data into the STGCN temporal graph convolutional network, distinguish high-risk behavioral features through graph attention mechanism, and output spatiotemporal feature data. Specifically, in this embodiment, a dynamic spatiotemporal graph is constructed based on fused multimodal feature data to input the STGCN temporal graph convolutional network. Using the joint coordinates of construction workers as core nodes, the feature information of corresponding joints from the fused multimodal feature data is allocated to each node, completing the feature assignment of the graph structure. Each worker on the construction site is treated as an independent individual, and the three-dimensional spatial coordinates of 18 key joints of the human body are extracted from visual data and point cloud data. Each joint coordinate is treated as an independent node, and the multimodal fusion features of corresponding joints from the fused multimodal feature data are allocated to each node as the node's feature information, completing the node definition. Two types of edges are constructed: one is a physical connection edge, which connects adjacent joint nodes of the same person according to the human physiological structure, representing the physical correlation of human actions; the other is a spatiotemporal correlation edge, which connects the same joint nodes of different people, representing the spatiotemporal interaction correlation between workers.

[0027] The STGCN graph attention mechanism calculates the attention weight of each node, assigning high attention weights to nodes and edges related to high-risk behaviors. The STGCN temporal convolution module extracts the temporal change features of nodes and edges, and the graph convolution module extracts spatial correlation features. The temporal and spatial features are then fused to output spatiotemporal feature data.

[0028] Multimodal feature data is fused and input into the STGCN temporal graph convolutional network. Using the joint coordinates of construction workers as core nodes, a dynamic spatiotemporal graph is constructed to model the spatiotemporal correlations of worker actions, equipment status, and environmental changes. Node definition: Each worker on the construction site is treated as an independent individual. The 3D spatial coordinates of 18 key joints of the human body are extracted from visual data and point cloud data, and each joint coordinate is treated as an independent node. Corresponding feature information is added to each node. Edge definition: Two types of edges are constructed to achieve spatiotemporal correlation modeling: one is physical connection edges, which connect adjacent joint nodes of the same person based on human physiological structure, representing the physical correlation of human actions; the other is spatiotemporal correlation edges, which connect the same joint node of different people, the same joint node of the same person at different times, and the joint nodes of personnel with surrounding equipment nodes. The weights of the spatiotemporal correlation edges are dynamically adjusted based on the spatiotemporal distance and feature similarity between nodes. Simultaneously, the node coordinates and edge relationships of the spatiotemporal graph are dynamically updated according to changes in the temporal data, achieving real-time capture of dynamic changes in the construction scene.

[0029] Based on the dynamic spatiotemporal graph, STGCN's graph attention mechanism weights and distinguishes the features of different nodes and edges, focusing on capturing spatiotemporal features related to high-risk behaviors. The graph attention mechanism automatically calculates the attention weight of each node, assigning higher attention weights to nodes and edges related to high-risk behaviors to highlight high-risk feature information and suppress redundant features in normal operation scenarios. Subsequently, STGCN's temporal convolution module extracts the temporal change features of nodes and edges, and the graph convolution module extracts spatial correlation features. The temporal and spatial features are then fused to output spatiotemporal feature data that accurately characterizes high-risk behaviors, equipment anomalies, and environmental hazards in construction scenarios. This data contains spatiotemporal correlation information and high-risk feature identifiers, providing core input for subsequent risk index calculations.

[0030] Step 104: Input the spatiotemporal feature data into the MANN memory-enhanced neural network, output the installation project risk index, evaluate based on the installation project risk index, and trigger the generation of a graded early warning intervention strategy.

[0031] Specifically, in this embodiment, the spatiotemporal feature data is input into the MANN memory-enhanced neural network. The memory module is used to store historical construction risk data, construction site safety standard thresholds, and risk assessment standard information for electromechanical installation projects. The calculation module compares the spatiotemporal feature data with the historical data in the memory module. Through similarity calculation, it matches the most similar historical risk cases, extracts the corresponding risk weights, and combines them with the high-risk indicators in the current spatiotemporal features to calculate the installation project risk index using a weighted summation method.

[0032] Spatiotemporal feature data is input into the MANN memory-enhanced neural network, which includes a memory module and a computation module. The memory module stores historical construction risk data, on-site safety standard thresholds, and risk assessment criteria for electromechanical installation projects. The computation module first compares the current spatiotemporal feature data with the historical data in the memory module. Through similarity calculation, it matches the most similar historical risk cases and extracts the corresponding risk weights. Subsequently, combined with high-risk markers in the current spatiotemporal features, a weighted summation method is used to calculate the real-time risk index of the current installation project. The risk index ranges from 0 to 100, with higher values ​​indicating higher safety risks in the current construction scenario. During the calculation process, the MANN network dynamically updates the data in the memory module, storing the feature data and risk index of the current construction scenario into the memory module, continuously optimizing the accuracy of subsequent risk index calculations.

[0033] Based on the calculated real-time risk index and the preset risk grading standards, a risk assessment is completed and a graded early warning is triggered. The preset risk grading standards are divided into four levels, as follows: Level 1 Risk: Risk index 0-20, no obvious safety hazards in the current construction scene, construction activities can proceed normally; Level 2 Risk: Risk index 21-40, minor safety hazards exist in the current construction scene, requiring a warning; Level 3 Risk: Risk index 41-70, significant safety hazards exist in the current construction scene, requiring a warning and preliminary control measures; Level 4 Risk: Risk index 71-100, major safety hazards exist in the current construction scene, requiring an immediate emergency warning and suspension of related construction activities. Based on risk assessment results and risk classification, the MANN network, combined with safety management rules and historical intervention cases in its memory module, automatically generates targeted intervention strategies. Each intervention strategy corresponds one-to-one with a risk level, ensuring precise control. Specific intervention strategies are as follows: Level 1 risk generates a routine control strategy, reminding on-site safety management personnel to strengthen daily inspections and provide safety reminders to construction workers; Level 2 risk generates a rectification reminder strategy, clarifying the responsible person and rectification content, and reassessing the risk index after rectification; Level 3 risk generates a control and rectification strategy, immediately dispatching safety management personnel to the site to stop violations, check equipment operating status, and implement temporary control measures, such as adjusting construction processes and suspending high-risk operations. Construction can only resume after rectification is completed and accepted; Level 4 risk generates an emergency control strategy, immediately suspending all construction activities in the relevant area, organizing the evacuation of construction workers to a safe area, and arranging professional personnel to investigate major safety hazards.

[0034] Its beneficial effects lie in breaking down data barriers by collecting four types of data—visual, point cloud, sensor, and management document data—and performing standardized preprocessing. This comprehensively covers safety information across all dimensions of the construction scenario, solving the problem of missed detections caused by the limited data in traditional methods. It improves the accuracy of risk identification by utilizing cross-modal alignment algorithms to achieve spatiotemporal synchronization and feature fusion of multiple data sources. Combined with the STGCN temporal graph convolutional network to capture spatiotemporal correlation features, and a graph attention mechanism to accurately distinguish high-risk behaviors, effectively reducing the probability of misjudgments and missed detections. It enables early risk prediction and tiered control by using the MANN memory-enhanced neural network combined with historical data to calculate a risk index, classifying risks into four levels and triggering corresponding warnings. It forms a closed-loop management system, automatically generating targeted intervention strategies and synchronously updating management documents, improving safety management efficiency, reducing labor costs, lowering the accident rate, and ensuring the personal and property safety of construction personnel.

[0035] Please see Figure 2In the safety management and early warning method for electromechanical installation engineering based on a large model, the dynamic spatiotemporal graph structure data constructed by fusing multimodal feature data is input into the STGCN temporal graph convolutional network. High-risk behavioral features are distinguished through the graph attention mechanism, and the spatiotemporal feature data is output. The steps include: Step 201: Input the fused multimodal feature data into the STGCN temporal graph convolutional network, and construct a dynamic spatiotemporal graph with the human joint coordinates of construction workers as the core nodes; Step 202: Taking each worker at the construction site as an independent individual, extract the three-dimensional spatial coordinates of 18 key joints of the human body from the visual data and point cloud data, treat each joint coordinate as an independent node, and add corresponding feature information to each node to define it as a node. Step 203: Construct two types of edges: physical connection edges, which connect adjacent joint nodes of the same person according to the human physiological structure, representing the physical correlation of human actions; and spatiotemporal connection edges, which connect the same joint nodes of different people.

[0036] The above describes embodiments of the present invention's method for safety management and early warning of electromechanical installation engineering based on a large model. Please refer to [link / reference]. Figure 3 In the large-scale model-based safety management early warning system for electromechanical installation projects, the system includes the following modules: The multimodal data acquisition module is used to acquire visual data, point cloud data, sensor data, and management document data from the construction site of electromechanical installation engineering, and to preprocess the acquired data to obtain multimodal data. The feature data fusion module is used to perform spatiotemporal synchronization and feature-level fusion of multimodal data using a cross-modal alignment algorithm to obtain fused multimodal feature data. The spatiotemporal feature extraction module is used to construct a dynamic spatiotemporal graph structure based on fused multimodal feature data. The data is input into the STGCN temporal graph convolutional network, and high-risk behavioral features are distinguished through the graph attention mechanism to output spatiotemporal feature data. The engineering safety management module is used to input spatiotemporal feature data into the MANN memory augmentation neural network, output the installation engineering risk index, evaluate the installation engineering risk index, and trigger graded early warning to generate intervention strategies.

[0037] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended technical solutions and equivalents.

Claims

1. A safety management and early warning method for electromechanical installation engineering based on a large model, characterized in that, The aforementioned safety management and early warning method for electromechanical installation projects includes the following steps: Acquire visual data, point cloud data, sensor data, and management document data from the construction site of electromechanical installation engineering, and preprocess the collected data to obtain multimodal data; The multimodal data is spatiotemporally synchronized and feature-level fused using a cross-modal alignment algorithm to obtain fused multimodal feature data; Based on the fusion of multimodal feature data, a dynamic spatiotemporal graph structure is constructed and input into the STGCN temporal graph convolutional network. High-risk behavioral features are distinguished through the graph attention mechanism, and spatiotemporal feature data is output. The spatiotemporal feature data is input into the MANN memory-enhanced neural network, which outputs an installation engineering risk index. The risk index is used for evaluation, and a graded early warning system is triggered to generate an intervention strategy.

2. The method for safety management and early warning of electromechanical installation engineering based on a large model as described in claim 1, characterized in that, The process involves acquiring visual data, point cloud data, sensor data, and management document data from the electromechanical installation construction site, preprocessing the collected data to obtain multimodal data, including: Acquire visual data, point cloud data, sensor data, and management document data from the construction site of electromechanical installation projects; Image enhancement algorithms are used to eliminate noise caused by lighting changes and dust occlusion in visual data. Frames are extracted from video data to retain key action frames. Human and equipment targets are located through target detection algorithms. After cropping invalid areas, the image size is unified. By using a pass-through filtering algorithm to remove environmental interference points and equipment scanning noise points from the point cloud data, the processed point cloud data is downsampled, and the point cloud data is uniformly converted into the global coordinate system of the construction site through coordinate calibration. Interpolation is used to fill in missing data from the sensor data, Kalman filtering is used to eliminate errors caused by sensor jitter, and the values ​​of different types of sensors are standardized. Based on NLP (Natural Language Processing) technology, unstructured text in management document data is segmented and stop word removed to extract information and convert it into structured data, resulting in multimodal data.

3. The method for safety management and early warning of electromechanical installation engineering based on a large model as described in claim 1, characterized in that, The method of using a cross-modal alignment algorithm to perform spatiotemporal synchronization and feature-level fusion of the multimodal data to obtain fused multimodal feature data includes: Based on the timestamps of sensor data, the frame times of visual data, the scanning times of point cloud data, and the update times of management document data are aligned to obtain time-aligned data. Using the global coordinate system of point cloud data in the time-aligned data as a reference, the pixel coordinates of the visual data are converted into spatial coordinates through camera calibration parameters, and accurately aligned with the spatial position of the point cloud data. At the same time, combined with the construction site zoning information in the management document data, various types of data are mapped to the corresponding construction areas to obtain spatiotemporally aligned data.

4. The method for safety management and early warning of electromechanical installation engineering based on a large model as described in claim 1, characterized in that, The method of using a cross-modal alignment algorithm to perform spatiotemporal synchronization and feature-level fusion of the multimodal data to obtain fused multimodal feature data includes: Human motion features, device appearance features, and environmental scene features are extracted from spatiotemporally aligned data using a CNN convolutional neural network. Spatial structure features, equipment installation accuracy features, and workspace spacing features were extracted from the spatiotemporal alignment data using the PointNet++ algorithm. The device operating status characteristics, personnel physiological characteristics, and environmental anomaly characteristics in the spatiotemporally aligned data are extracted using an MLP (Multilayer Perceptron). Safety rule features, risk threshold features, and personnel qualification features are extracted from spatiotemporally aligned data using the BERT large model. An attention-based cross-modal fusion algorithm is used to perform weighted fusion of various modal features. Weights are assigned according to the importance of each modal data under different construction scenarios to obtain fused multimodal feature data.

5. The method for safety management and early warning of electromechanical installation engineering based on a large model as described in claim 1, characterized in that, The dynamic spatiotemporal graph structure data constructed based on fused multimodal feature data is input into the STGCN temporal graph convolutional network. High-risk behavioral features are distinguished through a graph attention mechanism, and spatiotemporal feature data is output, including: Multimodal feature data is input into the STGCN temporal graph convolutional network, and a dynamic spatiotemporal graph is constructed with the human joint coordinates of construction workers as the core nodes. Taking each worker on the construction site as an independent individual, the three-dimensional spatial coordinates of 18 key joints of the human body are extracted from visual data and point cloud data. Each joint coordinate is treated as an independent node, and corresponding feature information is added to each node to define it as a node. Two types of edges are constructed: physical connection edges, which connect adjacent joint nodes of the same person based on human physiological structure, representing the physical correlation of human actions; and spatiotemporal connection edges, which connect the same joint nodes of different people.

6. The method for safety management and early warning of electromechanical installation engineering based on a large model as described in claim 1, characterized in that, The dynamic spatiotemporal graph structure data constructed based on fused multimodal feature data is input into the STGCN temporal graph convolutional network. High-risk behavioral features are distinguished through a graph attention mechanism, and spatiotemporal feature data is output, including: STGCN's graph attention mechanism calculates the attention weight of each node, assigning high attention weights to nodes and edges associated with high-risk behaviors. The temporal convolution module of STGCN is used to extract the temporal variation features of nodes and edges, and the graph convolution module is used to extract spatial correlation features. The temporal features and spatial features are then fused to output spatiotemporal feature data.

7. The method for safety management and early warning of electromechanical installation engineering based on a large model as described in claim 1, characterized in that, The process involves inputting the spatiotemporal feature data into the MANN memory-enhanced neural network, outputting an installation engineering risk index, evaluating the risk based on the index, and triggering a tiered early warning system to generate an intervention strategy. This includes: Spatiotemporal feature data are input into the MANN memory-enhanced neural network. The memory module is used to store historical construction risk data, construction site safety standard thresholds, and risk assessment standard information for electromechanical installation projects. The calculation module compares the spatiotemporal feature data with the historical data in the memory module. Through similarity calculation, it matches the most similar historical risk cases, extracts the corresponding risk weights, and combines them with the high-risk indicators in the current spatiotemporal features to calculate the installation project risk index using a weighted summation method.

8. A safety management and early warning system for electromechanical installation engineering based on a large model, characterized in that: The electromechanical installation engineering safety management and early warning system includes the following modules: The multimodal data acquisition module is used to acquire visual data, point cloud data, sensor data, and management document data from the construction site of electromechanical installation engineering, and to preprocess the acquired data to obtain multimodal data. The feature data fusion module is used to perform spatiotemporal synchronization and feature-level fusion of the multimodal data using a cross-modal alignment algorithm to obtain fused multimodal feature data; The spatiotemporal feature extraction module is used to construct a dynamic spatiotemporal graph structure based on fused multimodal feature data. The data is input into the STGCN temporal graph convolutional network, and high-risk behavioral features are distinguished through the graph attention mechanism to output spatiotemporal feature data. The engineering safety management module is used to input the spatiotemporal feature data into the MANN memory augmentation neural network, output the installation engineering risk index, evaluate the installation engineering risk index, and trigger graded early warning to generate intervention strategies.

9. The electromechanical installation engineering safety management and early warning system based on a large model as described in claim 8, characterized in that, The spatiotemporal feature extraction module includes the following sub-modules: The computation submodule is used to calculate the attention weight of each node through STGCN's graph attention mechanism, and assign high attention weights to nodes and edges related to high-risk behaviors. The extraction submodule is used to extract the temporal variation features of nodes and edges through the temporal convolution module of STGCN, extract spatial correlation features through the graph convolution module, fuse the temporal features and spatial features, and output spatiotemporal feature data.

10. The electromechanical installation engineering safety management and early warning system based on a large model as described in claim 8, characterized in that, The engineering safety management module includes the following sub-modules: The input submodule is used to input spatiotemporal feature data into the MANN memory-enhanced neural network. The memory module is used to store historical construction risk data of electromechanical installation projects, construction site safety specification thresholds, and risk assessment standard information. The evaluation submodule is used by the calculation module to compare the spatiotemporal feature data with the historical data in the memory module. Through similarity calculation, it matches the most similar historical risk cases, extracts the corresponding risk weights, and combines them with the high-risk indicators in the current spatiotemporal features to calculate the installation project risk index using a weighted summation method.

Images