A construction safety assessment method based on multi-model fusion

Abstract

The application relates to the technical field of construction safety assessment, and discloses a construction safety assessment method based on multi-model fusion. The method collects a multi-source safety monitoring data set of a construction site, covering image sequence data, sensor time sequence data and operation log text data; then parallel feature extraction processing is performed on the multi-source data to generate image feature vectors, sensor feature vectors and text feature vectors; then a multi-model fusion framework is called to perform cross-modal fusion processing on the three kinds of feature vectors to obtain a fusion safety feature set; safety risk assessment analysis is carried out based on the fusion safety feature set to generate a safety risk prediction value and a high-risk area identifier; finally, a safety strategy adjustment instruction is generated according to the risk prediction value and the high-risk area identifier, and is fed back to a field control system to trigger real-time intervention operation. The method can identify high-risk areas in time and trigger intervention, and provides intelligent support for safety management of a construction site.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of construction safety assessment, in particular to a construction safety assessment method based on multi-model fusion. BACKGROUND

[0002] In the field of building construction, safety management is an important link to protect the lives of construction personnel and avoid property losses. The construction site environment is complex and variable, involving a large number of personnel operations, large-scale mechanical equipment operation, and the construction of various temporary facilities. There are many potential safety risk points and they are hidden. Once a safety accident occurs, it often causes serious consequences. With the rapid development of the construction industry, the construction scale is continuously expanding, and the traditional safety assessment method gradually exposes many limitations.

[0003] Traditional construction safety assessment relies on manual inspection and single type of monitoring data, such as obtaining image information only through monitoring cameras or relying only on a few sensors to collect environmental parameters. This single data source assessment method has obvious shortcomings: image data can intuitively reflect the scene, but it is difficult to quantify environmental parameters such as temperature, humidity, equipment vibration, and other key information; sensor data can provide continuous physical quantity changes, but lack intuitive description of personnel behavior and equipment status in the scene; operation log text data records the construction process and operation specification execution, but it is difficult to associate and analyze with real-time scene information. Due to the limitations of different types of data, single data driven assessment methods are prone to problems such as incomplete risk identification, false positives or false negatives.

[0004] Existing safety assessment models mostly use single modal analysis methods, i.e. constructing an assessment model for a certain type of data, which is difficult to realize information complementation between different types of data. For example, an image-based recognition model may fail to identify personnel rule violations due to changes in lighting, obstructions, etc.; a sensor data-based warning model may misjudge the risk level due to a single parameter anomaly. At the same time, various types of data in the construction process exhibit massive and heterogeneous characteristics, and traditional methods have deficiencies in data processing efficiency and feature mining depth, making it difficult to meet the needs of real-time safety assessment.

[0005] With the development of intelligent monitoring technology, multiple types of monitoring equipment have been gradually deployed on construction sites, and a large amount of multi-source data has been accumulated. However, the existing technology lacks an effective cross-modal data fusion mechanism, and cannot fully exploit the correlation between different data. For example, there is a potential correlation between the sensor data of abnormal vibration of equipment, the image data of illegal operation of operators, and the illegal records in the operation log. However, traditional methods cannot analyze these information collaboratively, resulting in delayed risk warning or inaccurate evaluation. In addition, the existing evaluation methods cannot quickly generate targeted intervention strategies after risk identification, and cannot realize closed-loop management from risk perception to real-time control, making it difficult to effectively reduce the probability of accidents. Therefore, there is an urgent need for a construction safety evaluation method that can fuse multi-source data, accurately evaluate risks, and support real-time intervention. SUMMARY

[0006] The purpose of the present application is to provide a construction safety evaluation method based on multi-model fusion to solve the problems raised in the background art.

[0007] To achieve the above-mentioned purpose, the present application provides a construction safety evaluation method based on multi-model fusion, which comprises:

[0008] Collecting a set of multi-source safety monitoring data of the construction site, the set of multi-source safety monitoring data comprising image sequence data, sensor time series data, and operation log text data;

[0009] Performing parallel feature extraction processing on the set of multi-source safety monitoring data to generate image feature vectors, sensor feature vectors, and text feature vectors;

[0010] Calling a multi-model fusion framework to perform cross-modal fusion processing on the image feature vectors, sensor feature vectors, and text feature vectors to generate a set of fused safety features;

[0011] Performing safety risk evaluation analysis based on the set of fused safety features to generate safety risk prediction values and high-risk area identifiers;

[0012] Generating safety policy adjustment instructions based on the safety risk prediction values and high-risk area identifiers, and feeding back the safety policy adjustment instructions to the on-site control system to trigger real-time intervention operations.

[0013] Preferably, the set of multi-source safety monitoring data of the construction site comprises obtaining dynamic scene change information in the image sequence data, the image sequence data being derived from an array of cameras deployed in key construction areas;

[0014] Synchronously collecting vibration amplitude values, noise decibel values, and environmental temperature and humidity values in the sensor time series data, the sensor time series data being derived from an embedded sensor network;

[0015] Real-time record operation log text data of device operation records and personnel behavior descriptions in the operation log text data from the construction management database;

[0016] Integrate the dynamic scene change information, vibration amplitude value, noise decibel value, environment temperature and humidity value, device operation record and personnel behavior description into a time-aligned data set to generate a multi-source safety monitoring data set.

[0017] Preferably, the parallel feature extraction processing performed on the multi-source safety monitoring data set includes performing a spatial feature encoding operation on the image sequence data to generate an image feature vector, and the spatial feature encoding operation uses a convolutional neural network model to extract the spatial texture features of the image;

[0018] Performing a time-domain feature encoding operation on the sensor time series data to generate a sensor feature vector, and the time-domain feature encoding operation uses a long short-term memory network model to extract the time-domain fluctuation pattern;

[0019] Performing a semantic feature encoding operation on the operation log text data to generate a text feature vector, and the semantic feature encoding operation uses a pre-trained language model to extract key behavior semantics;

[0020] Parallelly performing the spatial feature encoding operation, the time-domain feature encoding operation and the semantic feature encoding operation, and ensuring that the timestamps of the output features are aligned.

[0021] Preferably, the calling of the multi-model fusion framework includes constructing a modal interaction attention mechanism to calculate a first cross-modal correlation weight between the image feature vector and the sensor feature vector, and simultaneously calculating a second cross-modal correlation weight between the text feature vector and the sensor feature vector.

[0022] Based on the first cross-modal correlation weight and the second cross-modal correlation weight, performing a weighted fusion operation on the image feature vector, the sensor feature vector and the text feature vector to generate an initial fusion feature;

[0023] Performing dimension reduction and normalization operations on the initial fusion feature to generate a fusion safety feature set, and the fusion safety feature set represents a comprehensive risk indicator of the multi-source safety monitoring data.

[0024] Preferably, the safety risk assessment analysis based on the fusion safety feature set includes inputting the fusion safety feature set into a pre-trained risk assessment model, and the risk assessment model includes a feature analysis layer and a risk prediction layer.

[0025] In the feature analysis layer, abnormal feature patterns and risk distribution rules in the fusion safety feature set are identified.

[0026] In the risk prediction layer, a safety risk prediction value is calculated according to the abnormal feature mode and the risk distribution rule, and a high-risk area identifier in the construction area is located;

[0027] The safety risk prediction value is a quantitative risk score, and the high-risk area identifier is a combination of a spatial coordinate and a risk level.

[0028] Preferably, the generation of the safety policy adjustment instruction according to the safety risk prediction value and the high-risk area identifier includes analyzing the spatial coordinate and the risk level in the high-risk area identifier, and matching the operation parameters of the corresponding construction link;

[0029] The safety risk prediction value is calculated to obtain a risk mitigation requirement value, and a dynamic adjustment strategy is generated in combination with the operation parameters;

[0030] The dynamic adjustment strategy includes a device operation parameter modification scheme, a personnel scheduling plan, and an environment control scheme;

[0031] The dynamic adjustment strategy is encoded into a safety policy adjustment instruction.

[0032] Preferably, the feedback of the safety policy adjustment instruction to the field control system to trigger real-time intervention operation includes transmitting the safety policy adjustment instruction to a field control terminal;

[0033] In the field control terminal, the device operation parameter modification scheme is decoded and applied to the mechanical control system, the personnel scheduling plan is decoded and applied to the scheduling system, and the environment control scheme is decoded and applied to the environment adjusting device;

[0034] The effect data of the real-time monitoring intervention operation is fed back to the multi-model fusion framework to update the feature extraction processing.

[0035] Preferably, the method further includes performing an incremental training update processing on the multi-model fusion framework, and the incremental training update processing uses the real-time monitored effect data as a training sample;

[0036] In the incremental training update processing, a deviation value of the effect data and the safety risk prediction value is calculated, and the weight parameters of the risk assessment model are adjusted based on the deviation value;

[0037] The updated risk assessment model is re-applied to the safety risk assessment and analysis step.

[0038] Preferably, the method further includes performing a historical optimization backtracking processing on the construction safety policy, and the historical optimization backtracking processing calls a historical safety policy adjustment instruction library;

[0039] Based on the strategy execution records in the historical safety policy adjustment instruction library, a strategy template with high effectiveness is screened out;

[0040] Adapt and optimize the strategy template in combination with the current security risk prediction value to generate an optimized security strategy adjustment instruction.

[0041] Preferably, the method further comprises generating a security assessment report output process that integrates the security risk prediction value, the high-risk area identification, and the security strategy adjustment instruction.

[0042] The integrated data is formatted into a visual report, including a risk heat map, a strategy execution timeline, and an intervention effect curve.

[0043] The visual report is output through the construction management platform to support decision-making.

[0044] Compared with the prior art, the present application has the following advantages:

[0045] The construction safety assessment method based on multi-model fusion provided by the present application realizes all-around perception of construction safety risks by collecting multi-source safety monitoring data of the construction site. The method incorporates image sequence data, sensor time series data, and operation log text data, covering multiple dimensions such as visual scene information, physical environment parameters, and process operation records, breaking the limitations of traditional single data evaluation methods. Different types of data reflect the construction state from different angles, image data can capture intuitive scene features such as personnel behavior and equipment position, sensor data can record quantitative information such as environmental temperature and humidity and equipment operating parameters, and text data contains construction process compliance, operation specification execution, and other content. The collaborative collection of multi-source data enables safety assessment to cover more potential risk scenarios and reduces the risk of missing data due to missing data.

[0046] In the feature extraction stage, parallel processing is used to generate image, sensor, and text feature vectors respectively. This processing method can fully preserve the unique properties of each modality data. The image feature vector can accurately depict the visual details in the scene, such as whether personnel are wearing safety equipment and whether equipment is in a compliant position. The sensor feature vector can effectively extract the variation law of physical quantities, such as equipment vibration frequency anomalies and environmental concentration exceeding standards. The text feature vector can extract key information from operation logs, such as illegal operation records and process deviation descriptions. Parallel extraction ensures that each feature vector can fully retain the core information of the corresponding data, laying a foundation for subsequent fusion analysis and avoiding the problem of mutual interference of different modal information in the single feature extraction process, thereby improving the effectiveness and relevance of feature data.

[0047] Through the cross-modal fusion processing by the multi-model fusion framework, the internal correlation between different types of feature vectors can be deeply mined, and a more rich fusion safety feature set can be generated. The image features, sensor features and text features are not isolated, for example, the image features of personnel not operating according to the specification may be correlated with the device abnormal operation features recorded by the sensor and the illegal operation records in the text log. The cross-modal fusion analyzes these correlation relationships through model cooperation, and integrates the scattered feature information into a fusion feature with a global perspective. This fusion process not only retains the advantages of each modal feature, but also discovers hidden risk patterns that are difficult to identify by single modal analysis, such as the superimposed risk of device parameter abnormality and operation process violation, thereby improving the richness and representation ability of safety features.

[0048] The safety risk assessment analysis based on the fusion safety feature set can more accurately generate safety risk prediction values and high-risk area identification. The fusion feature set contains multi-dimensional and multi-correlation information, so that the risk assessment model can comprehensively consider various influencing factors and reduce the evaluation deviation caused by the one-sidedness of a single feature. For example, when evaluating the risk of a certain area, not only the image display of the area is referred to, but also the environmental parameters in the sensor data and the historical violation records in the text log are combined to verify the risk level from multiple angles, so that the risk prediction value is more consistent with the actual construction state, and the identification of high-risk areas is more accurate, which can accurately locate the areas or links that need to be paid attention to.

[0049] In the process of generating safety policy adjustment instructions and feeding back to the field control system, the rapid connection from risk assessment to real-time intervention is realized. Based on the accurate risk prediction value and clear high-risk area identification, the generated policy adjustment instructions are more targeted and can directly point to the risk source, such as issuing warning instructions for personnel violation areas and triggering shutdown inspection instructions for device abnormal areas. This immediate feedback mechanism quickly translates safety assessment results into actual control actions, shortens the time interval from risk identification to intervention execution, enables the construction site to respond to potential risks in a timely manner, avoids risk accumulation and expansion, and improves the dynamics and timeliness of construction safety management.

[0050] In addition, this method adapts to the complex and variable environmental characteristics of the construction site, and through multi-source data fusion and multi-model cooperation, it can cope with complex situations such as changes in light, device interference and non-standard text records, reducing the influence of environmental factors on the evaluation results. At the same time, this method reduces the dependence on manual assessment, improves the efficiency and consistency of safety assessment through automated data collection, feature extraction, fusion analysis and instruction generation, and enables safety management to cover the entire construction process, thereby building an intelligent safety management mechanism for the construction site from data perception, analysis and evaluation to real-time intervention. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A timing diagram of the construction safety assessment method based on multi-model fusion according to the present application;

[0052] Figure 2 A flowchart of the multi-source safety monitoring data collection process;

[0053] Figure 3 A multi-source safety monitoring data collection and analysis diagram;

[0054] Figure 4 A flowchart of multi-source data parallel feature extraction;

[0055] Figure 5 A multi-source feature extraction and fusion process analysis diagram;

[0056] Figure 6 A flowchart of safety risk assessment analysis. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0058] Please refer to Figure 1 The present application provides a construction safety assessment method based on multi-model fusion, which comprises:

[0059] By integrating multi-source heterogeneous data on the construction site, a cross-modal feature fusion framework is constructed to realize dynamic assessment and real-time intervention of safety risks. First, a camera array, an embedded sensor network and a construction management database are deployed to synchronously collect image sequence data, sensor time series data and operation log text data, forming a time-aligned multi-source safety monitoring data set. Spatial features, time domain features and semantic features are extracted in parallel by using convolutional neural networks, long short-term memory networks and pre-trained language models, and cross-modal feature weighted fusion is realized through a modal interaction attention mechanism. The safety feature set after fusion is input into a risk assessment model to output a quantitative risk score and high-risk area spatial coordinates, and finally safety policy adjustment instructions including equipment parameter adjustment, personnel scheduling and environmental control schemes are generated to drive the on-site control system to complete the closed-loop intervention.

[0060] Embodiment 1: Please refer to Figure 2, the collection process of the multi-source safety monitoring data set of the construction site, which is realized by deploying multiple types of sensing devices and data integration systems. The high-definition camera array arranged in the key construction area adopts a combination of fixed installation and pan-tilt control, covering high-risk areas such as tower crane operation area, deep foundation pit, and material storage area. Each camera is equipped with an anti-shake bracket and an automatic cleaning device to adapt to the vibration and dust environment of the construction site. The video acquisition module sets the H.265 encoding format to reduce bandwidth requirements while ensuring image quality, and transmits to the edge computing node through optical fiber network and 5G dual channel. The metadata record of image sequence data includes device ID, geographic location, timestamp, and shooting angle, which facilitates subsequent spatial feature correlation analysis.

[0061] The deployment of the embedded sensor network adopts a hierarchical topology structure, and vibration sensors are installed in key stress parts such as tower crane base, support scaffold, and foundation pit support structure, realizing omnidirectional vibration monitoring through three-axis acceleration measurement. Noise sensors are arranged in areas with concentrated construction machinery and worker operation surfaces, and windshields are used to reduce airflow interference, with a sound pressure level measurement range covering typical construction noise spectrum. Temperature and humidity sensors are distributed in semi-closed construction areas and material storage areas, and some nodes are equipped with anti-radiation covers to avoid the influence of direct sunlight. All sensor nodes form a low-power wide-area network through the LoRaWAN protocol, and gateway devices are deployed at high points in the construction site to achieve stable communication within a radius of 500 meters. The sensor data acquisition module has a built-in Kalman filter algorithm to eliminate transient interference signals and output smooth time series data streams.

[0062] The construction management database adopts a distributed architecture, and real-time access to the tower crane black box system records device operation parameters such as lifting weight, amplitude, and rotation angle, as well as personnel positioning system provided access records and activity tracks. Unstructured text data is converted into paper inspection records through OCR technology, combined with behavior description information filled in by the supervisor's mobile terminal, to form a complete operation log data set. The database sets data verification rules to automatically mark abnormal values and missing fields, triggering manual review processes. The time synchronization system uses GPS clock source and NTP server redundancy configuration to ensure that the time deviation of all data sources is controlled within milliseconds.

[0063] The multi-source data integration platform is built on Apache Kafka, setting up three independent message topics: image data, sensor data, and text data. Each topic is configured with a corresponding partitioning strategy and retention period. The stream processing engine consumes data from each topic in real time, performs time window alignment operations, and packages heterogeneous data within the same time period into data blocks. The data blocks are stored in Avro serialization format, containing a unified timestamp index, data body, and quality identifier. In the data quality control stage, a multi-level verification mechanism is set up, including image blur detection, sensor signal transition recognition, and text sensitive word filtering. Abnormal data is transferred to a repair queue for interpolation or removal.

[0064] The data storage system employs a layered design. Hot data is stored in the Alluxio memory acceleration layer, supporting high-concurrency real-time queries; warm data is written to a distributed file system, organized by project segment and time range; cold data is archived to object storage, with a configured lifecycle management strategy for automatic migration. The storage format is columnar Parquet files, with appropriate compression encoding used for different data types: image metadata uses dictionary encoding, sensor values ​​use Delta encoding, and text data uses Snappy compression. The data access interface provides both RESTful API and JDBC methods, supporting multi-dimensional retrieval by device ID, time range, and spatial region.

[0065] A comprehensive data security system is implemented throughout the entire data acquisition process. Video stream transmission employs end-to-end TLS encryption, sensor data is digitally signed to prevent tampering, and database operations are logged with complete audit logs. Access control is based on the RBAC model, differentiating permissions for different roles such as device acquisition accounts, data analysis accounts, and administrator accounts. The data backup strategy follows a 3-2-1 principle, maintaining three copies: a local disk, a local disaster recovery center, and off-site cloud storage. Incremental backups are performed daily, and full backups are performed weekly.

[0066] The system's operational status monitoring module tracks key indicators such as camera online rate, sensor battery level, and database connection count in real time, triggering tiered alarms when thresholds are exceeded. The maintenance and management terminal displays a geographical distribution map of each device and supports remote configuration parameter updates and firmware upgrades. The data acquisition frequency is dynamically adjusted according to the construction phase, automatically increasing the sampling rate during high-risk operation periods such as concrete pouring, and operating in energy-saving mode during normal phases.

[0067] The embodiment builds a stable and reliable multi-source safety monitoring data acquisition system through fine equipment selection and deployment, strict data quality control, and perfect system architecture design. The complementary configuration of various types of sensing devices eliminates monitoring blind spots, the time synchronization mechanism ensures the correlation of cross-modal data, and the streaming processing architecture meets the real-time requirements. The hierarchical storage scheme takes into account access performance and cost efficiency, and comprehensive security measures guarantee data integrity and privacy. The system monitoring function realizes intelligent operation and maintenance of acquisition devices, providing a high-quality data foundation for subsequent feature extraction and risk assessment. The entire acquisition process is highly adaptable to the construction site environment and can continuously and stably output time-aligned multi-source safety monitoring data sets.

[0068] Referring to Figure 3 , the acquisition process of multi-source safety monitoring data in the construction site is shown. The left side of the chart shows the distribution of image acquisition devices in different areas of the construction site, including tower crane operation area, deep foundation pit, material storage area, and other high-risk areas. Each point represents a camera position, and the size of the point represents the amount of data collected by the device. The shape of the point distinguishes the device type. The right side of the chart contains two subplots: the upper subplot shows the trend of vibration sensor data over time, with peaks indicating abnormal vibration events during construction; the lower subplot shows the decibel value changes collected by the noise sensor and the temperature and humidity sensor data, reflecting the noise level and changes in the monitoring construction environment during different construction stages. These data collectively form a multi-source safety monitoring data set, providing a foundation for subsequent feature extraction and risk assessment.

[0069] Example 2: Referring to Figure 4 , the parallel feature extraction and cross-modal fusion processing process of multi-source safety monitoring data. This process realizes efficient feature extraction and deep fusion of image sequence data, sensor time series data, and operation log text data by building a multi-model collaborative computing framework. The image feature extraction module uses an improved ResNet-50 architecture, removing the global average pooling layer of the original model and retaining the spatial feature map output by the convolution layer. After batch normalization processing, the input image sequence is sequentially compressed through a 1x1 convolution kernel, and a 2048-dimensional feature tensor is output at the fifth convolution stage. The time dimension feature aggregation uses an attention weighted pooling method to calculate the motion correlation between consecutive frames, generating an image feature vector with time sequence perception.

[0070] The sensor time series data processing module is constructed based on a bidirectional LSTM network. The input layer receives standardized vibration, noise, and temperature and humidity three-channel data. The network hidden layer uses a gated recurrent unit structure, with 128 neural units to capture time domain dependencies. The feature extraction process introduces residual connections to alleviate the gradient vanishing problem of deep networks. The output layer applies a temporal attention mechanism to highlight the feature contributions of abnormal fluctuations, and finally generates a 64-dimensional sensor feature vector. The network training uses a curriculum learning strategy, gradually increasing the input sequence length to improve the model's ability to model long-term dependencies.

[0071] The text feature extraction module loads a pre-trained BERT-base model and performs incremental training on the construction safety domain corpus. The input text is processed by adding special markers after tokenization, and the embedding layer outputs a 768-dimensional word vector sequence. The Transformer encoder is stacked with 12 layers to capture the semantic associations between operation records and behavior descriptions through self-attention mechanisms. The feature representation uses the [CLS] marker corresponding to the hidden state, which is projected through a fully connected layer to maintain the original dimension. The model fine-tuning stage uses an adversarial training method to enhance the ability to adapt to the diversity of construction terminology.

[0072] The parallel computing framework is built on the PyTorch ecosystem, with a dedicated data loader to synchronize the reading of multi-modal inputs. The computation graph optimization uses dynamic shape inference techniques to allow automatic adjustment of the computation path for different batches of input data. GPU resource management uses CUDA streams to implement kernel-level parallelism, with image convolution, time series loops, and text attention calculations performed on separate stream processors. Memory allocation uses a unified virtual address space to avoid frequent data copying. The feature output buffer has a timestamp verification mechanism to ensure strict alignment of multi-modal features in the time dimension.

[0073] The cross-modal fusion process uses a hierarchical attention architecture, with the first interaction layer handling the association between image and sensor features. A cross-attention module is constructed to calculate the relevance scores between image spatial positions and sensor time points, generating a 128-dimensional weight matrix. The second interaction layer handles the association between text and sensor features, using a multi-head attention mechanism to calculate the alignment degree of semantics and time series on 8 attention heads, outputting a 64-dimensional weight representation. The feature weighting process introduces a gating mechanism to dynamically adjust the contribution proportion of each modality, avoiding information suppression. The initial features after fusion are processed by layer normalization, with a skip connection to preserve the original feature information.

[0074] The dimensionality reduction processing stage realizes the combination of principal component analysis and nonlinear transformation. First, 1024-dimensional features are compressed to 512-dimensional features by PCA, retaining 95% of the variance information. Then, a two-layer bottleneck autoencoder is applied, using a ReLU activation function for nonlinear mapping, and finally outputting 256-dimensional fusion safety features. Feature normalization uses an improved batch normalization method, and the statistics calculation considers the data distribution within the time sliding window. The output feature set is accompanied by quality evaluation indicators, including inter-modal consistency score and feature stability measure.

[0075] Quality control of the computing process runs throughout the entire processing flow. The image feature extraction stage sets up a receptive field verification module to detect the spatial coverage of the feature map. The time series feature extraction implements outlier robust processing using the Huber loss function to reduce noise interference. The text feature extraction adds a syntax tree verification to ensure the logical reasonableness of the semantic representation. The fusion process monitors inter-modal conflict indicators, and when the threshold is exceeded, triggers the feature recalculation process. System resource monitoring tracks parameters such as GPU utilization, memory occupancy, and computing delay in real time, and dynamically adjusts the batch size to maintain stable throughput.

[0076] The model updating mechanism supports the combination of online learning and offline training. The image feature extractor uses momentum contrast learning for continuous optimization. The sensor feature model is updated through time series prediction tasks, and the text feature model is incrementally trained regularly to incorporate new construction specification terminology. The fusion module parameters are adjusted using a meta-learning strategy, automatically optimizing the number of attention heads and weight calculation methods based on historical fusion results. All model updates are implemented through containerized deployment for seamless switching, and version management uses a blue-green release strategy to ensure service continuity.

[0077] This embodiment realizes efficient feature extraction and deep correlation of multi-source heterogeneous data through a carefully designed parallel computing architecture and hierarchical fusion strategy. The improved deep learning model fully exploits the characteristics of each modality data, and the attention mechanism effectively captures cross-modal interaction patterns. Strict quality control measures ensure the reliability of feature extraction, and flexible updating mechanisms enable the system to have continuous evolution capability. The optimization design of the computing process fully utilizes hardware acceleration resources to meet the real-time requirements of the construction site. The output fusion safety feature set fully retains the risk indication information of multi-source data, providing high representation capability input features for subsequent safety risk assessment. The entire processing flow is highly adaptable to the construction safety monitoring scene, achieving a good balance between computing efficiency and feature quality.

[0078] Referring to Figure 5This chart demonstrates the parallel feature extraction and cross-modal fusion processing of multi-source security monitoring data. The left side of the chart shows the feature extraction results for three modalities: image features (showing the spatial feature distribution extracted from image sequences, with point density reflecting feature strength); sensor features (showing the temporal feature value distribution, with point positions reflecting the projection of feature vectors across different dimensions); and text features (presenting the clustering distribution of semantic features extracted from operation logs). The right side of the chart shows the cross-modal fusion results: the upper subplot shows the dimensionality-reduced projection of the fused features, with point density reflecting the concentration of fused features in different regions; the lower subplot shows the stability index of the fused features, with higher values ​​indicating more reliable features. This chart comprehensively illustrates the transformation process from multi-source heterogeneous data to a unified security feature set, providing high-quality input for risk assessment.

[0079] Example 3: See Figure 6 This process integrates risk assessment and analysis with safety strategy generation, incorporating a set of integrated safety features. By constructing a hierarchical risk assessment model and an intelligent strategy generation system, it achieves quantitative analysis and dynamic control of construction safety risks. The risk assessment model employs an ensemble learning framework, taking a 256-dimensional fused safety feature vector as input. After hierarchical processing through a feature parsing layer and a risk prediction layer, it outputs actionable safety strategy instructions.

[0080] The feature parsing layer comprises 20 gradient boosting decision trees, each with a maximum depth of 7 layers, using a weighted Gini coefficient as the node splitting criterion. During decision tree training, a feature importance evaluation metric is introduced to calculate the contribution of each feature dimension to risk prediction. An abnormal feature pattern recognition module monitors the deviation of feature values, marking a feature component as an abnormal pattern when it meets the following conditions:

[0081]

[0082] in, This represents the anomaly degree of the j-th dimension feature. For the current eigenvalue, This is the historical sliding window average. The standard deviation is the sliding window value. Risk distribution pattern analysis employs a density clustering algorithm to identify high-risk clustering areas in a three-dimensional projected space, generating a distribution heatmap containing spatial coordinates and risk intensity.

[0083] The risk prediction layer integrates the results of multiple sub-models, including random forest regressor, support vector machine, and deep neural network. The prediction results of each sub-model are fused through a dynamic weighting method, and the weight coefficients are self-adaptively adjusted according to the recent prediction accuracy of the model. The safety risk prediction value is normalized to the interval [0, 1], and the calculation process considers context factors such as construction phase, operation type, and environmental conditions. The high-risk area identification generation module marks areas with prediction values exceeding the threshold as different risk levels, with red warning corresponding to a risk value above 0.9, orange warning corresponding to the interval 0.7-0.9, and yellow warning corresponding to the interval 0.5-0.7. Spatial coordinate positioning uses BIM model mapping technology to map risk points in the feature space to GPS locations in the actual construction scene.

[0084] The strategy generation system analyzes the risk assessment results and implements differentiated strategy generation for different types of high-risk areas. The safety strategy for tower crane operation areas includes three dimensions: torque limiter parameter adjustment, rotation speed limitation, and lifting path optimization. The torque limiter parameter adjusts the safety margin linearly according to the risk prediction value, and sets a dynamic threshold to prevent overload. The rotation speed limitation strategy considers the current wind speed and load weight to calculate the maximum allowed rotation speed. The lifting path optimization uses an improved A* algorithm to avoid personnel-intensive areas and high-risk orientations.

[0085] The safety strategy for deep foundation pit areas includes three aspects: support structure monitoring, personnel evacuation, and drainage control. The support structure monitoring strategy adjusts the monitoring frequency according to the vibration characteristic value, and starts real-time deformation monitoring in high-risk state. The personnel evacuation strategy calculates the optimal evacuation path and collection point position, and guides in real time through the positioning system. The drainage control strategy predicts seepage risk based on temperature and humidity characteristics, and dynamically adjusts the water pump operating parameters.

[0086] The environmental regulation strategy generation module handles the intervention scheme for temperature and humidity abnormal areas. In high-temperature areas, the spray cooling system is started, and the spray intensity is proportional to the temperature deviation value. In high-humidity areas, the dehumidification equipment is activated, and the operating parameters are fed forward adjusted according to the humidity prediction curve. In noise exceeding areas, mechanical sound insulation schemes and operation time adjustment suggestions are generated to balance construction progress and environmental protection requirements.

[0087] The strategy encoding module converts the generated safety measures into standardized control instructions. Device parameter adjustment instructions use ModbusRTU protocol format, including register address, data type, and set value. Personnel scheduling instructions generate JSON format task list and push to mobile terminal for execution. Environmental control instructions are converted into BACnet object identifiers and attribute values, and transmitted through building automation protocol. All instructions are attached with timestamp and version number to ensure the timing consistency and traceability of execution.

[0088] The instruction priority management system sorts the strategies according to the risk level and influence range, and the red warning instruction obtains the highest transmission priority. The instruction verification module simulates the execution effect and predicts the risk change trend after the implementation of the strategy, and modifies the strategy that may have a negative impact. The instruction distribution system adopts the publish / subscribe mode, and routes the strategy instruction to the corresponding execution terminal through the message queue.

[0089] The strategy feedback mechanism collects the instruction execution state and device response data in real time, and establishes a strategy effect evaluation matrix. The execution delay monitoring records the time difference from the generation to the effectiveness of the instruction, and the timeout triggers the activation of the standby strategy. The effect tracking module compares the risk characteristic changes before and after the implementation of the strategy, and calculates the strategy effectiveness index. The adaptive adjustment system dynamically updates the strategy generation parameters according to the feedback data, and optimizes the quality of subsequent decision-making.

[0090] This embodiment realizes the accurate identification and effective intervention of construction safety risks through a hierarchical risk assessment model and an intelligent strategy generation system. The feature analysis layer deeply excavates the risk indication mode in the fused features, and the risk prediction layer outputs reliable evaluation results by combining the advantages of multiple algorithms. The strategy generation process considers the actual construction constraints to generate executable safety control measures. The instruction coding and transmission mechanism ensures the accurate landing of the strategy, and the feedback cycle realizes the continuous optimization of the system. The entire processing flow forms a complete closed loop from risk perception to intervention implementation, maintaining the real-time and adaptability of safety control in complex construction environments. The collaborative design of risk assessment and strategy generation enables the system to identify micro-feature abnormalities and develop macro-control schemes, meeting the multi-dimensional needs of modern large-scale engineering construction safety management.

[0091] Example 4: The real-time execution of safety strategy adjustment instructions and the dynamic updating process of the model realize the adaptive optimization of construction safety management through the construction of a closed-loop control system. Taking the tower crane safety control in a high-rise building project as an example, when the system detects that the risk prediction value of the tower crane operation area exceeds the threshold value, the complete intervention process is implemented in the following manner:

[0092] The safety strategy adjustment instructions received by the field control terminal contain multi-dimensional control parameters, and the specific contents of a typical tower crane control instruction set are shown in the following table.

[0093] Table 1: Tower crane safety strategy adjustment instruction parameter table

[0094] Parameter category Parameter item Original setting value Adjustment value Effective condition Operating parameter Maximum rotation speed 0.8 r / min 0.5 r / min Wind speed > 8 m / s and load > 3 t Lifting acceleration limit 0.5 m / s2 0.3 m / s2 Risk value > 0.7 Safety device Torque limit threshold 85% 75% Vibration amplitude > 0.2 g Anti-collision distance 5m 8m Peripheral person density > 3 persons / 100 m2 Monitoring setting Video analysis frequency 1 Hz 5 Hz Risk level = red Sensor sampling rate 10 Hz 20 Hz Temperature > 35 °C

[0095] The instruction transmission system converts the above parameters into PLC recognizable function codes through the industrial Internet of Things gateway, wherein the operating parameters are written into the holding register 40001-40010 address segment, and the safety device parameters are written into the coil register 00001-00008 address bit. The transmission process adopts a CRC-16 check mechanism, each data packet contains a time stamp and an instruction sequence number, ensuring the integrity and timing correctness of the transmission.

[0096] When the tower crane control system performs parameter adjustment, it first enters a safety verification state: the main controller compares the difference between the new and old parameters, and triggers the manual confirmation process when the amplitude exceeds the safety margin; the secondary parameter adjustment is directly written into the running memory and takes effect in real time. During the execution process, the black box recorder synchronously stores the parameter change log, including the modification time, operator ID and effective state. The mechanical response monitoring module collects real-time operating data after adjustment, including the current rotation speed, actual lifting acceleration and other physical quantities, and performs deviation analysis with the set values.

[0097] The instruction execution of the personnel dispatching system is reflected in the dynamic adjustment of the electronic fence system. When the system generates personnel evacuation instructions for high-risk areas, the positioning base station immediately updates the virtual boundary of the electronic fence through the following process: the UWB positioning tag receives a low-frequency wake-up signal, and the area personnel are prompted to evacuate; the access controller removes the door lock restriction on the escape path; the broadcast system plays directional voice guidance. The dispatching effect is visualized through real-time personnel position heat map, and the number of remaining personnel in the area is counted every minute.

[0098] The strategy execution of the environmental control equipment presents a stepwise adjustment feature. Taking the spray dust suppression system as an example, when the PM2.5 sensor value and the risk prediction model output jointly trigger the adjustment condition, the control system operates according to the following steps: first, start the water pump to reach the basic pressure, then open the electromagnetic valve in stages according to the risk level of each area, and give priority to the upwind direction and high-risk operation areas. The flow sensor monitors the working state of each nozzle, and automatically switches to the standby pipeline in abnormal conditions. The environmental improvement effect is evaluated by multi-sensor fusion, including comprehensive feedback from dust concentration monitor, humidity sensor and camera visibility analysis.

[0099] The model updating system adopts an online learning mechanism to collect state data after strategy execution to form training samples. Each sample contains the comparison of parameters before and after adjustment, actual response data and changes in the final risk indicators. The feature weight adjustment module analyzes the contribution of each dimension feature to the prediction accuracy and reduces the weight of features with long-term errors. The model version management adopts an AB testing strategy, and the new version model is first tested on 10% of the devices, and then widely promoted after comparing the prediction effect stability.

[0100] The abnormal handling mechanism is activated in the following typical scenarios: when the actual rotation speed of the tower crane cannot reach the set value for 3 minutes, the system judges that there may be mechanical failure, automatically issues a degraded operation instruction and triggers a maintenance work order; when the area density does not decrease after the personnel evacuation instruction is executed, the standby evacuation path calculation is started and pushed to the terminal of the site management personnel; when the response delay of the environmental conditioning equipment exceeds the threshold, switch to the redundant control system and mark the fault node.

[0101] The feedback data acquisition system constructs a multi-channel information collection network, including device operation state interface, personnel positioning data stream and environmental sensor time series database. Data cleaning link handles transmission packet loss, sensor drift and abnormal peak value problems, and uses sliding window filtering algorithm to ensure data quality. The effect evaluation index calculation strategy changes the risk value rate, response time and resource consumption before and after execution, and forms a comprehensive performance score.

[0102] This embodiment constructs a safety management system that adapts to complex construction environment through fine instruction decomposition execution, multi-system collaborative control and continuous learning optimization. The specific implementation process of the tower crane control as an example shows that the system can convert abstract risk prediction value into specific device parameter adjustment, and verify the execution effect through a rigorous feedback mechanism. The linkage implementation of personnel scheduling and environmental control shows the overall coordination ability of the system, and the model updating mechanism guarantees the adaptability of long-term operation. The whole closed-loop control process shows reliable execution precision and timely response speed in the real construction scene, meeting the strict requirements of modern engineering projects on safety management real-time and accuracy. The whole-chain data record from instruction generation to effect feedback provides a complete information foundation for the continuous improvement of construction safety management.

[0103] Example 5: Historical optimization backtracking and evaluation report generation process of construction safety strategy, by constructing a knowledge base driven strategy optimization system and a visual report system, realizing the continuous improvement and information transparency of safety management decision. The historical safety strategy adjustment instruction library adopts time series database storage structure, records the complete life cycle information of each strategy instruction, including generation time, risk feature mode, specific control parameter, execution effect index and operator annotation metadata. Data modeling uses event trace mode, preserves the full state change trajectory of the strategy from generation to abolition, and supports strategy scene restoration at any time point.

[0104] The policy retrieval system establishes a multi-dimensional index structure, the spatial index is organized based on the R-tree algorithm, and supports query by the geographical range of the construction area; the time index uses a segmented hash table to realize millisecond-level time window retrieval; the risk feature index uses an approximate nearest neighbor algorithm to quickly locate similar risk patterns in historical cases. The retrieval process introduces a fuzzy matching mechanism to handle feature deviation caused by differences in construction environments, and filters low-relevance policy records by setting a similarity threshold.

[0105] The policy effectiveness evaluation module analyzes the execution records in the historical database and calculates multiple performance indicators for each policy template. The duration indicator reflects the stability of the policy, the modification frequency indicator reflects the adaptability of the policy, the risk reduction rate indicator measures the direct effect of the policy, and the resource consumption indicator evaluates the economy of the policy. The evaluation results are converted into standardized scores, and the balance relationship in each dimension is displayed in the form of a radar chart to help identify policy templates with excellent comprehensive performance.

[0106] The template adaptation optimization process uses the genetic algorithm idea to optimize the parameters of the selected high-quality strategies. The control parameters are encoded as gene sequences, the advantages of different strategies are combined through crossover operation, and random disturbance is introduced through mutation operation to explore new parameter combinations. The fitness function considers the current risk characteristics and construction progress requirements to generate the most suitable parameter configuration for the actual site needs. The optimization process sets an iteration termination condition, and when the improvement amplitude of the optimal solution is less than a certain threshold for three consecutive generations, the final scheme is automatically output.

[0107] The safety assessment report generation system uses a modular document architecture, including four core chapters: risk overview, regional analysis, policy tracking, and trend prediction. The risk overview chapter uses a three-dimensional pie chart to show the distribution proportion of different levels of risk, and a dynamic text summary to explain the overall safety situation. The regional analysis chapter superimposes the construction site plan and risk heat map, uses a gradient color scale to represent the risk gradient, and clicks on a specific area to drill down to view detailed monitoring data.

[0108] The policy tracking chapter establishes a timeline visualization interface, with the horizontal axis representing time progress and the vertical axis displaying the implementation trajectory of each type of policy. The equipment control strategy is represented by a blue trajectory line, the personnel scheduling strategy is represented by a green trajectory line, and the environment adjustment strategy is represented by an orange trajectory line. Each trajectory line is labeled with key decision points, and hovering displays the policy parameters and effect indicators at that time. The trend prediction chapter uses a time series analysis algorithm to draw a risk change curve for the next three days, and uses a confidence interval to represent the prediction uncertainty.

[0109] The report interaction function supports multi-dimensional data exploration, providing risk level filtering, time range selection, and device type screening, etc. The view linkage mechanism ensures data consistency between different chapters. When the user selects a specific area in the risk heat map, the policy tracking chapter automatically focuses on displaying the policy implementation records of that area. The report annotation system allows users to add text comments and problem markers. These user-generated contents together with the original data form a complete decision support information.

[0110] The document output system provides various format conversion functions. The PDF version maintains a fixed layout for archiving and printing, the HTML5 version retains interactive elements for online browsing, and the JSON format is used for data exchange between systems. The version control system records the differences of each report generation, supporting historical version tracing and comparison. The access control system is based on role-based permission management, limiting the report content and operation permissions that users of different levels can view.

[0111] The automatic push mechanism distributes reports according to user roles and responsibility ranges. Project managers receive complete reports, safety officers get risk detail chapters, and device administrators receive relevant control policy sections. The push channels include email systems, mobile application messages, and web platform notifications. Important warning information triggers additional SMS reminders. The reading status tracking system records the opening time and browsing duration of users, and sends a second reminder for key reports that have not been viewed in time.

[0112] This implementation builds a self-evolving safety decision support system by deeply integrating historical policy data and current risk characteristics. The policy optimization process is not simply a historical replication, but an intelligent parameter exploration combined with genetic algorithms, allowing excellent strategies to adapt to changing construction conditions. The visual report system breaks through the limitations of traditional static documents and helps managers understand the safety situation through interactive data exploration. From policy backtracking to report generation, the complete process forms a closed-loop learning mechanism for safety management decisions, enabling the system to continuously accumulate experience and knowledge, and continuously improve safety control levels. The entire implementation process embodies the concept of data-driven decision-making, transforming scattered policy implementation records into systematic decision-making knowledge, providing a traceable, verifiable, and optimized scientific management tool for construction safety management.

[0113] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and implementations, it is to be understood that the terminology used is for the purpose of descriptive clarity and that it should be taken in its broadest possible sense. For example, the terms "a", "an", and "the" include both singular and plural referents unless the context clearly dictates otherwise. The terms "comprises", "comprising", "includes", "including" and the like can be used in conjunction with the term "consisting of to include the elements or steps listed after such conjunctive language, but not to the exclusion of other elements or steps. The singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise.

[0114] While the embodiments of the application have been shown and described herein, it is understood that modifications, substitutions, changes, and alterations can be made by those skilled in the art without departing from the spirit and scope of the present application, which is defined by the appended claims and their equivalents.

Claims

1.A construction safety assessment method based on multi-model fusion, characterized in that, The method comprises collecting a multi-source safety monitoring data set of a construction site, the multi-source safety monitoring data set comprising image sequence data, sensor time series data and operation log text data; Parallel feature extraction processing is performed on the multi-source safety monitoring data set to generate image feature vectors, sensor feature vectors and text feature vectors; A multi-model fusion framework is called to perform cross-modal fusion processing on the image feature vectors, sensor feature vectors and text feature vectors to generate a fused safety feature set; Based on the fused safety feature set, safety risk assessment analysis is performed to generate a safety risk prediction value and a high-risk area identifier; A safety policy adjustment instruction is generated according to the safety risk prediction value and the high-risk area identifier, and the safety policy adjustment instruction is fed back to a field control system to trigger real-time intervention operations; The parallel feature extraction processing on the multi-source safety monitoring data set comprises a spatial feature encoding operation performed on the image sequence data to generate the image feature vectors, the spatial feature encoding operation using a convolutional neural network model to extract spatial texture features of the image; A time-domain feature encoding operation is performed on the sensor time series data to generate the sensor feature vectors, the time-domain feature encoding operation using a long short-term memory network model to extract time-domain fluctuation patterns; A semantic feature encoding operation is performed on the operation log text data to generate the text feature vectors, the semantic feature encoding operation using a pre-trained language model to extract key behavior semantics; The spatial feature encoding operation, the time-domain feature encoding operation and the semantic feature encoding operation are executed in parallel, and the timestamps of the output features are aligned; The calling of the multi-model fusion framework to perform cross-modal fusion processing on the image feature vectors, sensor feature vectors and text feature vectors comprises constructing a modal interaction attention mechanism to calculate a first cross-modal correlation weight between the image feature vectors and the sensor feature vectors, and simultaneously calculating a second cross-modal correlation weight between the text feature vectors and the sensor feature vectors; Based on the first cross-modal correlation weight and the second cross-modal correlation weight, a weighted fusion operation is performed on the image feature vectors, sensor feature vectors and text feature vectors to generate initial fused features; Dimensionality reduction and normalization operations are performed on the initial fused features to generate the fused safety feature set, which represents a comprehensive risk indicator of the multi-source safety monitoring data. 2.The construction safety assessment method based on multi-model fusion according to claim 1, characterized in that, The collection of the multi-source safety monitoring data set of the construction site comprises obtaining dynamic scene change information in the image sequence data, the image sequence data being sourced from an array of cameras deployed in key construction areas; Vibration amplitude values, noise decibel values and environmental temperature and humidity values in the sensor time series data are synchronously collected, the sensor time series data being sourced from an embedded sensor network; Device operation records and personnel behavior descriptions in the operation log text data are recorded in real time, the operation log text data being sourced from a construction management database; The dynamic scene change information, vibration amplitude values, noise decibel values, environmental temperature and humidity values, device operation records and personnel behavior descriptions are integrated into a time-aligned data set to generate the multi-source safety monitoring data set. 3.The construction safety assessment method based on multi-model fusion according to claim 1, characterized in that, The safety risk assessment analysis based on the fusion safety feature set includes inputting the fusion safety feature set into a pre-trained risk assessment model, which includes a feature analysis layer and a risk prediction layer; In the feature analysis layer, abnormal feature patterns and risk distribution rules in the fusion safety feature set are identified; In the risk prediction layer, a safety risk prediction value is calculated according to the abnormal feature patterns and the risk distribution rules, and a high-risk area identifier in the construction area is located; The safety risk prediction value is a quantitative risk score, and the high-risk area identifier is a combination of spatial coordinates and risk levels. 4.The construction safety assessment method based on multi-model fusion according to claim 3, characterized in that, The safety policy adjustment instruction generated according to the safety risk prediction value and the high-risk area identifier includes analyzing the spatial coordinates and risk levels in the high-risk area identifier and matching the operation parameters of the corresponding construction link; Based on the safety risk prediction value, a risk mitigation requirement value is calculated, and a dynamic adjustment strategy is generated in combination with the operation parameters; The dynamic adjustment strategy includes a device operation parameter modification scheme, a personnel scheduling plan, and an environment control scheme; The dynamic adjustment strategy is encoded into a safety policy adjustment instruction. 5.The construction safety assessment method based on multi-model fusion according to claim 4, characterized in that, The safety policy adjustment instruction is fed back to the field control system to trigger real-time intervention operations, including transmitting the safety policy adjustment instruction to the field control terminal; In the field control terminal, the device operation parameter modification scheme is decoded and applied to the mechanical control system, the personnel scheduling plan is decoded and applied to the scheduling system, and the environment control scheme is decoded and applied to the environment regulation equipment; The effect data of the real-time intervention operation is monitored in real time, and the effect data is fed back to the multi-model fusion framework to update the feature extraction process. 6.The construction safety assessment method based on multi-model fusion according to claim 5, characterized in that, The method further includes performing an incremental training update process on the multi-model fusion framework, using the real-time monitored effect data as training samples; In the incremental training update process, the deviation value of the effect data and the safety risk prediction value is calculated, and the weight parameters of the risk assessment model are adjusted based on the deviation value; The updated risk assessment model is re-applied to the safety risk assessment analysis step. 7.The construction safety assessment method based on multi-model fusion according to claim 6, characterized in that, The method further includes performing a historical optimization backtracking process on the construction safety policy, and the historical optimization backtracking process calls a historical safety policy adjustment instruction library; Based on the policy execution records in the historical safety policy adjustment instruction library, a strategy template with high effectiveness is selected; The strategy template is adapted and optimized in combination with the current safety risk prediction value to generate an optimized safety policy adjustment instruction. 8.The construction safety assessment method based on multi-model fusion according to claim 7, characterized in that, The method further includes a safety assessment report output process, which integrates the safety risk prediction value, the high-risk area identifier, and the safety policy adjustment instruction; The integrated data is formatted into a visual report, including a risk heat map, a strategy execution timeline, and an intervention effect curve; The visual report is output through a construction management platform to support decision-making.

Images