Construction site safety behavior identification method based on deep learning

Abstract

The invention relates to the technical field of behavior recognition, in particular to a construction site safety behavior recognition method based on deep learning, which comprises the steps of collecting environmental data of a construction site in real time, preprocessing the environmental data to improve data quality and diversity, extracting feature vectors, recognizing specific behaviors and identities of construction site personnel, and predicting personnel behaviors. Potential dangerous behaviors are identified, multi-level risk assessment is carried out according to risk levels, after high-risk behaviors are identified, the system not only triggers an alarm, but also automatically takes intervention measures, and through linkage with construction site mechanical equipment, operation of dangerous equipment is automatically stopped so as to prevent accident expansion. According to the invention, through multi-modal data acquisition and fusion processing, the accuracy of behavior identification is improved, refined multi-level risk management is realized, the system can rapidly trigger alarm and automatically take intervention measures, and the emergency processing efficiency and the overall safety level of a construction site are significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of behavior recognition technology, and in particular to a construction site safety behavior recognition method based on deep learning. Background Art

[0002] Safety management has always been an important and complex issue in the construction site environment. Traditional construction site safety management methods usually rely on manual inspections and regular checks, which is not only time-consuming and labor-intensive, but also difficult to achieve all-weather, all-round monitoring due to human resource limitations, resulting in safety hazards that are difficult to detect and deal with in a timely manner. With the expansion of construction site scale and the complexity of construction processes, the limitations of this traditional method have become increasingly obvious, and it cannot meet the needs of modern construction sites for efficient and safe management.

[0003] However, the inventors have discovered that this technical solution still has at least the following defects: First, data collection and processing are not comprehensive, and the accuracy of behavior recognition is poor: the existing technology usually relies on only a single type of sensor, such as a camera, for data collection, lacks a comprehensive perception of the diversity and complexity of the environment, resulting in insufficient data representativeness. At the same time, the data processing method is simple, and fails to fully utilize multimodal data for comprehensive analysis, resulting in poor accuracy and robustness of behavior recognition. In contrast, the present invention can comprehensively and accurately reflect the environmental conditions of the construction site through multimodal data collection (including cameras, infrared sensors, temperature sensors) and multimodal fusion processing, thereby improving the accuracy and reliability of behavior recognition.

[0004] Second, risk assessment lacks hierarchical management, making it difficult to achieve refined control: Most existing risk assessment methods use a single risk judgment standard and lack hierarchical management. Differentiated response measures cannot be taken for behaviors of different risk levels, resulting in inadequate management measures and affecting safety management effectiveness. The present invention proposes a multi-level risk assessment method, which can achieve refined risk management by quantitatively evaluating the identified behaviors and potentially dangerous behaviors and classifying them according to preset risk level thresholds, and take corresponding response measures for different risk levels, thereby improving the effectiveness of construction site safety management.

[0005] Third, emergency response relies on manual intervention and has a slow response speed: the traditional emergency response mechanism mainly relies on manual intervention, and has the problems of slow response speed and low processing efficiency. In an emergency, it is difficult to take effective measures in time, which can easily lead to the expansion of accidents. After identifying high-risk behaviors, the system of the present invention can not only trigger an alarm, but also automatically take intervention measures. By linkage with the construction site machinery and equipment, it automatically stops the operation of dangerous equipment to prevent the expansion of accidents. This automated emergency response mechanism greatly improves the system's response speed and processing efficiency, and significantly improves the overall safety level of the construction site. Summary of the invention

[0006] Based on the above purpose, the present invention provides a construction site safety behavior recognition method based on deep learning. The construction site safety behavior recognition method based on deep learning includes the following steps: S1, data collection: collect environmental data of the construction site in real time through environmental perception sensors (including cameras, infrared sensors, and temperature sensors) distributed throughout the construction site; S2, data preprocessing: perform multimodal fusion processing on different types of environmental data, associate and integrate them, and preprocess the environmental data after multimodal fusion processing, including data cleaning, normalization and data augmentation; S3, feature extraction: use convolutional neural network to extract features from the preprocessed environmental data and extract feature vectors representing the behavior of construction site personnel; S4, individual behavior recognition and behavior correlation analysis: Use the extracted feature vectors to identify the specific behaviors of construction site personnel, including normal behaviors, abnormal behaviors, and potentially dangerous behaviors. Use biometric recognition technology to identify the identities of construction site personnel and correlate individual behaviors with their historical behavior data; S5, behavior prediction: combining historical behavior data and environmental change trends to predict human behavior and identify potential dangerous behaviors; S6, multi-level risk assessment: Based on the results of individual behavior identification and association analysis as well as the identification of potential dangerous behaviors, the identified behaviors and potential dangerous behaviors are subject to multi-level risk assessment according to risk levels; S7, emergency response and automatic intervention: After identifying high-risk behaviors, the system not only triggers an alarm, but also automatically takes intervention measures.

[0007] Optionally, S1 includes: S11, sensor placement: Environment perception sensors are placed in key areas of the construction site, including entrances, construction areas, danger zones, and rest areas. The number and placement of sensors are adjusted according to the scale and layout of the construction site. S12, camera data collection: using a camera to capture video data of the construction site in real time; S13, infrared data collection: using infrared sensors to collect thermal imaging data on the construction site; S14, temperature data collection: Use temperature sensors to collect ambient temperature data at various locations on the construction site in real time.

[0008] Optionally, S2 includes: S21, multimodal data fusion: different types of environmental data are synchronized in time and integrated into a unified data set. The weighted average method is used to perform multimodal fusion of different types of data, which is expressed as: ; in, For at time point The fusion characteristics of , and is the fusion weight coefficient, , and are the features of video, infrared and temperature data respectively; S22, data cleaning: filtering the fused data and removing the noise using the median filter algorithm; S23, normalization processing: normalizing different types of data; S24, data enhancement: enhance the data by random cropping, rotation, scaling and translation.

[0009] Optionally, S3 includes: S31, data input: input the preprocessed environmental data into the convolutional neural network (CNN), and the input data is expressed as: ; in, is the input data set, For time point The fusion characteristics of S32, convolution layer: The input data is convolved through multiple convolution layers to extract low-level features. Each convolution layer uses multiple convolution kernels (filters) to perform feature mapping on the input data, which is expressed as: ; in, Output the pixel value in the feature map of the convolutional layer, is the pixel value of the input data, is the weight of the convolution kernel, is the bias term, is the spatial location of the input data, is the size of the convolution kernel, is the index of the convolution kernel; S33, Activation function: A non-linear activation function is applied after each convolutional layer. ReLU In order to introduce nonlinear characteristics, it can be expressed as: ; in, is the activation function output, is the output of the convolutional layer; Pooling layer: Max pooling is applied after the convolutional layer for dimensionality reduction and feature compression, expressed as: ; in, is the output of the pooling layer, is the input feature map, is the pooling step size, is the pooling window size; Fully connected layer: The features extracted by the convolution layer and the pooling layer are integrated and mapped to a high-dimensional feature space through the fully connected layer to generate a feature vector representing the behavior of the construction site personnel, which is expressed as: ; in, is the output feature vector, is the weight matrix of the fully connected layer, Output features of the pooling layer, is the bias term; Feature vector output: The final generated feature vector is expressed as: ; in, is the feature vector representing the behavior of construction site personnel, is the dimension of the feature vector.

[0010] Optionally, the S4 includes: S41, behavior classification: The extracted feature vectors are input into the pre-trained Extreme Gradient Boosting (XGBoost) model. The Extreme Gradient Boosting (XGBoost) model processes the feature vectors to identify the specific behaviors of the construction site personnel, including normal behaviors (such as walking, standing), abnormal behaviors (such as falling, running) and potentially dangerous behaviors (such as entering restricted areas, not wearing safety equipment, etc.); S42, biometric identification: Identify the identity of site personnel through biometric identification technology; S43, behavior association analysis: associate the identified individual behavior with its historical behavior data to establish a behavior pattern library; S44, behavior pattern update: based on the real-time monitored behavior data, the behavior pattern library is continuously updated and improved.

[0011] Optionally, the S41 includes: S411, feature vector input: the preprocessed feature vector Input into the pre-trained Extreme Gradient Boosting (XGBoost) model; S412, Model training and optimization: Construction of tree structure: In each iteration, the extreme gradient boosting (XGBoost) model updates the model by building a new decision tree to fit the residual of the current model, which is expressed as: ; in, For the The predicted value of a tree, Leaf Node The weight of is the indicator function, indicating the sample Is it a leaf node? ; S413, optimization objective function: The optimization objective function of the extreme gradient boosting (XGBoost) model includes an error term and a regularization term, which is expressed as: ; in, is the objective function, is the loss function, is the true value, is the predicted value, For the The regularization term of the tree; S414, Regularization term: The regularization term is used to control the complexity of the model and is expressed as: ; in, is the weight of the number of leaf nodes in the tree, is the number of leaf nodes, is the weight of the L2 regularization term, is the leaf node weight; S415, behavior classification: After each iteration, the extreme gradient boosting (XGBoost) model accumulates the updated prediction results with the previous prediction results and finally predicts the specific behavior type, which is expressed as: ; in, For sample The final predicted value of For the The prediction results of the tree, is the number of iterations.

[0012] Optionally, S5 includes: S51, integration of historical behavior data and environmental data: historical behavior data and environmental data are time-aligned and integrated into a time series data set, expressed as: ; in, is a time series dataset, For time point behavioral characteristics, For time point environmental characteristics; S52, feature extraction and preprocessing: extract and preprocess the integrated time series data to generate a high-dimensional feature vector that can reflect the characteristics of the time series, expressed as: ; S53, Variational Autoencoder (VAE) training: Train the feature vector using the variational autoencoder (VAE) model; S54, Future Behavior Prediction: Predict the behavior characteristics at a future time point, expressed as: ; in, For the predicted future time point behavioral characteristics, is the current time point; S55, Dangerous Behavior Identification: Analyze the predicted behavioral characteristics to identify potential dangerous behaviors.

[0013] Optionally, the S53 includes: S531, encoder: use the encoder to input feature vector Mapping to latent variables in latent space , generating the mean vector and the logarithmic variance vector , expressed as: ; in, is the encoder function, is the mean of the latent variable, is the log variance of the latent variable; S532, Reparameterization Technique: Use the reparameterization technique to sample latent variables from the latent space , expressed as: ; in, is the noise of standard normal distribution; S533, decoder: Use the decoder to convert the latent variables Mapping back to feature space, reconstructing the input feature vector , expressed as: ; in, is the decoder function, is the reconstructed feature vector; S534, loss function: The model is trained by optimizing the loss function of the variational autoencoder. The loss function includes reconstruction error and KL divergence, which is expressed as: ; in, is the reconstruction error, is the KL divergence, is the posterior distribution generated by the encoder, is the prior distribution.

[0014] Optionally, S6 includes: S61, integration of behavior identification and association analysis results: integrating the behaviors and potential dangerous behaviors identified in the individual behavior identification and association analysis steps; S62, integration of dangerous behavior identification results: integrating the potential dangerous behavior results identified in the behavior prediction step; S63, Risk level assessment: Based on the identified behaviors and potentially dangerous behaviors, assess the risk level of each behavior; S64, multi-level risk classification: the assessed risk levels are classified according to the preset risk level thresholds.

[0015] Optionally, the S7 includes: S71, High-risk behavior identification: Identify high-risk behaviors in the multi-level risk assessment step; S72, alarm triggering: when high-risk behavior is detected, the system triggers an alarm; S73, automatic intervention measures: The system automatically takes intervention measures to stop the operation of dangerous equipment by linking with the construction site machinery and equipment; S74, linkage control: linkage between the system and the construction site machinery and equipment, realizing automatic stop of the equipment through control instructions; S75, emergency plan initiated: After the alarm is triggered and automatic intervention measures are taken, the system initiates the emergency plan and notifies relevant personnel to handle it.

[0016] Beneficial effects of the present invention: The present invention uses multimodal data collection and fusion processing, and comprehensively utilizes multiple environmental sensing devices such as cameras, infrared sensors, and temperature sensors to comprehensively capture environmental data on the construction site. The convolutional neural network (CNN) is used to extract features from the pre-processed multimodal data, which can more accurately identify the specific behaviors of construction site personnel, including normal behaviors, abnormal behaviors, and potentially dangerous behaviors. Compared with the traditional method of single sensor data, the present invention significantly improves the accuracy and robustness of behavior recognition and reduces the probability of misjudgment and missed judgment.

[0017] The present invention proposes a multi-level risk management method based on quantitative evaluation, which integrates the identified behaviors and potentially dangerous behaviors and evaluates the risk level of each behavior. The risk levels of behaviors and dangerous behaviors are classified using quantitative formulas, and multi-level classification is performed according to preset risk level thresholds, realizing refined management from low risk to high risk. Different risk levels correspond to different management strategies, ensuring that the most appropriate response measures are taken for each behavior, thereby improving the accuracy and effectiveness of construction site safety management.

[0018] The system of the present invention can quickly trigger an alarm and automatically take intervention measures. By linking with the mechanical equipment on the construction site, the system can automatically stop the operation of dangerous equipment to prevent the accident from escalating. At the same time, the system activates the emergency plan and notifies relevant personnel to handle it. The automated emergency response mechanism significantly improves the reaction speed and processing efficiency. Compared with the traditional method that relies on manual intervention, it greatly reduces the response time and enhances the overall safety and emergency handling capabilities of the construction site. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 Schematic diagram of a construction site safety behavior identification method based on deep learning according to an embodiment of the present invention; Figure 2 Schematic diagram of individual behavior recognition and behavior association analysis according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0022] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0023] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0024] like Figure 1-Figure 2 As shown, the construction site safety behavior recognition method based on deep learning includes the following steps: S1, data collection: collect environmental data of the construction site in real time through environmental perception sensors (including cameras, infrared sensors, and temperature sensors) distributed throughout the construction site; S2, data preprocessing: perform multimodal fusion processing on different types of environmental data, associate and integrate them to improve the accuracy and robustness of behavior recognition, and preprocess the environmental data after multimodal fusion processing, including data cleaning, normalization and data enhancement, to improve data quality and diversity; S3, feature extraction: use convolutional neural network (CNN) to extract features from the preprocessed environmental data and extract feature vectors representing the behavior of construction site personnel; S4, individual behavior identification and behavior association analysis: Use the extracted feature vectors to identify the specific behaviors of construction site personnel, including normal behaviors (such as walking, standing), abnormal behaviors (such as falling, running) and potentially dangerous behaviors (such as entering restricted areas, not wearing safety equipment, etc.). Use biometric recognition technology to identify the identity of construction site personnel and associate individual behaviors with their historical behavior data for analysis; S5, behavior prediction: combining historical behavior data and environmental change trends to predict human behavior and identify potential dangerous behaviors; S6, multi-level risk assessment: Based on the results of individual behavior identification and association analysis as well as the identification of potential dangerous behaviors, the identified behaviors and potential dangerous behaviors are subject to multi-level risk assessment according to risk levels; S7, emergency response and automatic intervention: After identifying high-risk behaviors, the system not only triggers an alarm, but also automatically takes intervention measures. By linking with the construction site machinery and equipment, it automatically stops the operation of dangerous equipment to prevent the accident from expanding.

[0025] Through multi-sensor data collection and preprocessing, the use of deep learning technology for feature extraction and behavior recognition, combined with the correlation analysis of individual behaviors and multi-level risk assessment, real-time monitoring and prediction of the behavior of construction site personnel is achieved, and emergency response and intervention measures are automatically triggered in high-risk situations, which improves the accuracy and efficiency of construction site safety management, reduces safety hazards, and ensures the safety of construction site personnel.

[0026] S1 includes: S11, sensor placement: environmental perception sensors are placed in key areas of the construction site, including entrances, construction areas, danger zones, and rest areas. The number and placement of sensors are adjusted according to the scale and layout of the construction site to ensure that the sensor coverage of each area can monitor the overall situation of the construction site and the comprehensiveness and accuracy of data collection; S12, camera data acquisition: use the camera to capture the video data of the construction site in real time at a rate of 30 frames per second with a resolution of 1080p to ensure clear images; S13, infrared data acquisition: using infrared sensors to collect thermal imaging data of the construction site at a rate of 10 frames per second, generating thermal maps for monitoring at night and in low light conditions; S14, temperature data collection: Use temperature sensors to collect ambient temperature data at various locations on the construction site in real time, and record the data once per minute to ensure the real-time nature of the temperature data.

[0027] By placing environmental perception sensors in key areas of the construction site to ensure the comprehensiveness and accuracy of data collection, it is able to capture video, thermal imaging and temperature data in real time and adapt to different environmental conditions, ensuring the real-time and accuracy of the data. This multi-source data collection method has greatly improved the efficiency and reliability of on-site monitoring of the construction site, and provided a comprehensive environmental data foundation to facilitate subsequent behavior identification and safety management.

[0028] S2 includes: S21, multimodal data fusion: different types of environmental data are synchronized in time and integrated into a unified data set. The weighted average method is used to perform multimodal fusion of different types of data, which is expressed as: ; in, For at time point The fusion characteristics of , and is the fusion weight coefficient, , and are the features of video, infrared and temperature data respectively; S22, data cleaning: noise filtering is performed on the fused data, and the median filter algorithm is used to remove the noise, which is expressed as: ; in, For time point After cleaning the data, is the median filter operation, to is the original data within the time window; S23, normalization: normalize different types of data to make them have the same scale, which is convenient for subsequent feature extraction and analysis, expressed as: ; in, is the normalized data, is the original data, and are the minimum and maximum values ​​of the data respectively; S24, data enhancement: Enhance the data by randomly cropping, rotating, scaling, and translating to increase data diversity and prevent overfitting. Specifically, it includes: S241, random cropping, is expressed as: ; in, is the cropped image, is the original image, is the coordinate of the clipping starting point, The height and width of the cropping window; S242, rotation, is expressed as: ; in, is the rotated image, is the rotation operation function, is the rotation angle; S243, scaling, is expressed as: ; in, is the scaled image, is the scaling function, is the scaling factor; S244, translation, is expressed as: ; in, is the image after translation, is the translation operation function, For and The translation distance in the direction.

[0029] Through multimodal data fusion, data cleaning, normalization and data enhancement, the effective integration and high-quality processing of different types of environmental data are ensured, which significantly improves the accuracy and robustness of behavior recognition and provides a solid data foundation for subsequent feature extraction and behavior recognition.

[0030] S3 includes: S31, data input: input the preprocessed environmental data into the convolutional neural network (CNN), and the input data is expressed as: ; in, is the input data set, For time point The fusion characteristics of S32, convolution layer: The input data is convolved through multiple convolution layers to extract low-level features. Each convolution layer uses multiple convolution kernels (filters) to perform feature mapping on the input data, which is expressed as: ; in, Output the pixel value in the feature map of the convolutional layer, is the pixel value of the input data, is the weight of the convolution kernel, is the bias term, is the spatial location of the input data, is the size of the convolution kernel, is the index of the convolution kernel; S33, Activation function: A non-linear activation function is applied after each convolutional layer. ReLU In order to introduce nonlinear features and improve the expressive power of the model, it can be expressed as: ; in, is the activation function output, is the output of the convolutional layer; Pooling layer: After the convolutional layer, the maximum pooling is applied for dimensionality reduction and feature compression, reducing the amount of data and computational complexity while retaining important features, expressed as: ; in, is the output of the pooling layer, is the input feature map, is the pooling step size, is the pooling window size; Fully connected layer: The features extracted by the convolution layer and the pooling layer are integrated and mapped to a high-dimensional feature space through the fully connected layer to generate a feature vector representing the behavior of the construction site personnel, which is expressed as: ; in, is the output feature vector, is the weight matrix of the fully connected layer, Output features of the pooling layer, is the bias term; Feature vector output: The final generated feature vector is expressed as: ; in, is the feature vector representing the behavior of construction site personnel, is the dimension of the feature vector; By using convolutional neural networks to perform multi-level convolution, activation, pooling and full connection operations on the preprocessed environmental data, high-dimensional feature vectors representing the behavior of construction site personnel are effectively extracted. The generated feature vectors provide a reliable data basis for subsequent behavior recognition and analysis, enhancing the intelligence and accuracy of construction site safety management.

[0031] S4 includes: S41, behavior classification: The extracted feature vectors are input into the pre-trained Extreme Gradient Boosting (XGBoost) model. The Extreme Gradient Boosting (XGBoost) model processes the feature vectors to identify the specific behaviors of the construction site personnel, including normal behaviors (such as walking, standing), abnormal behaviors (such as falling, running) and potentially dangerous behaviors (such as entering restricted areas, not wearing safety equipment, etc.); S42, biometric identification: The identity of the construction site personnel is identified through biometric identification technology, expressed as: ; in, For the identified person’s identity, is the biometric recognition function, is biometric data; S43, behavior association analysis: associate the identified individual behavior with its historical behavior data and establish a behavior pattern library, which is expressed as: ; in, For time point Behavioral history data, is the correlation analysis function, For personnel identity, A collection of historical behavior data of the person; S44, behavior pattern update: based on the real-time monitored behavior data, the behavior pattern library is continuously updated and improved to ensure that the system can more accurately identify and predict human behavior, expressed as: ; in, For the behavior pattern library, For the old behavior pattern library, For new behaviors and historical data.

[0032] By using a pre-trained convolutional neural network model to process the extracted feature vectors, the specific behaviors of construction site personnel can be accurately identified, a behavior pattern library can be established, and the behavior pattern library can be updated in real time to ensure that the system can more accurately identify and predict personnel behavior, thereby improving the accuracy of behavior recognition and the robustness of the system, and enhancing the intelligence and effectiveness of construction site safety management.

[0033] S41 includes: S411, feature vector input: the preprocessed feature vector Input into the pre-trained Extreme Gradient Boosting (XGBoost) model; S412, Model training and optimization: Construction of tree structure: In each iteration, the extreme gradient boosting (XGBoost) model updates the model by building a new decision tree to fit the residual of the current model, which is expressed as: ; in, For the The predicted value of a tree, Leaf Node The weight of is the indicator function, indicating the sample Is it a leaf node? ; S413, optimization objective function: The optimization objective function of the extreme gradient boosting (XGBoost) model includes an error term and a regularization term, which is expressed as: ; in, is the objective function, is the loss function, is the true value, is the predicted value, For the The regularization term of the tree; S414, Regularization term: The regularization term is used to control the complexity of the model and avoid overfitting, expressed as: ; in, is the weight of the number of leaf nodes in the tree, is the number of leaf nodes, is the weight of the L2 regularization term, is the leaf node weight; S415, behavior classification: After each iteration, the extreme gradient boosting (XGBoost) model accumulates the updated prediction results with the previous prediction results and finally predicts the specific behavior type, which is expressed as: ; in, For sample The final predicted value of For the The prediction results of the tree, is the number of iterations.

[0034] By inputting the preprocessed feature vector into the pre-trained extreme gradient boosting (XGBoost) model and optimizing the objective function, the specific behaviors of construction site personnel can be accurately identified, the accuracy of behavior classification is improved, and model overfitting is avoided through effective regularization, which improves the stability and robustness of the model, thereby enhancing the intelligence and effectiveness of construction site safety management.

[0035] S5 includes: S51, integration of historical behavior data and environmental data: historical behavior data and environmental data are time-aligned and integrated into a time series data set, expressed as: ; in, is a time series dataset, For time point behavioral characteristics, For time point environmental characteristics; S52, feature extraction and preprocessing: extract and preprocess the integrated time series data to generate a high-dimensional feature vector that can reflect the characteristics of the time series, expressed as: ; S53, Variational Autoencoder (VAE) training: Train the feature vector using the variational autoencoder (VAE) model; S54, Future Behavior Prediction: Predict the behavior characteristics at a future time point, expressed as: ; in, For the predicted future time point behavioral characteristics, is the current time point; S55, Dangerous behavior identification: Analyze the predicted behavior characteristics and identify potential dangerous behaviors, expressed as: ; in, For time point dangerous behavior, Identify functions for dangerous behaviors.

[0036] Through the above steps, the system can combine historical behavior data and environmental change trends to effectively predict people's future behavior and identify potential dangerous behaviors, thereby improving the accuracy of behavior predictions. It can also identify and prevent potential safety hazards in advance, enhance the foresight and reliability of construction site safety management, provide intelligent safety management methods for construction sites, and significantly improve the overall safety level of construction sites.

[0037] S53 includes: S531, encoder: use the encoder to input feature vector Mapping to latent variables in latent space , generating the mean vector and the logarithmic variance vector , expressed as: ; in, is the encoder function, is the mean of the latent variable, is the log variance of the latent variable; S532, Reparameterization Technique: Use the reparameterization technique to sample latent variables from the latent space , ensuring that the model can be trained end-to-end, expressed as: ; in, is the noise of standard normal distribution; S533, decoder: Use the decoder to convert the latent variables Mapping back to feature space, reconstructing the input feature vector , expressed as: ; in, is the decoder function, is the reconstructed feature vector; S534, loss function: The model is trained by optimizing the loss function of the variational autoencoder. The loss function includes reconstruction error and KL divergence, which is expressed as: ; in, is the reconstruction error, is the KL divergence, is the posterior distribution generated by the encoder, is the prior distribution; By using the optimized variational autoencoder (VAE) to train feature vectors, we can effectively capture complex patterns in the data, improve the training efficiency of the model, and enhance the accuracy of predictions of future behaviors, providing smarter and more reliable support for construction site safety management.

[0038] S6 includes: S61, integration of behavior identification and association analysis results: Integrate the behaviors and potential dangerous behaviors identified in the individual behavior identification and association analysis steps, expressed as: ; in, is the set of identified behaviors, For the individual behaviors; S62, integration of dangerous behavior identification results: Integrate the potential dangerous behavior results identified in the behavior prediction step, expressed as: ; in, For the set of potentially dangerous behaviors identified, For the Potentially dangerous behavior; S63, Risk level assessment: Based on the identified behaviors and potentially dangerous behaviors, assess the risk level of each behavior, expressed as: ; in, For behavior and risky behaviors The risk level, and The weights of behaviors and risky behaviors, and score the severity of behaviors and risky behaviors; S64, multi-level risk classification: the assessed risk level is classified according to the preset risk level threshold, expressed as: = in, is the risk level, Score the risk, and Thresholds for classifying risk levels.

[0039] By integrating individual behavior and potentially dangerous behavior data, accurately assessing the risk level of each behavior and classifying them according to preset risk level thresholds, comprehensive and accurate risk management is achieved, the accuracy and coverage of risk assessments are improved, and through hierarchical management and timely response, site safety strategies and resource allocation are optimized, significantly improving the efficiency and effectiveness of site safety management.

[0040] S7 includes: S71, High-risk behavior identification: High-risk behaviors are identified in the multi-level risk assessment step, expressed as: ; in, A collection of high-risk behaviors. For the high-risk behavior; S72, alarm triggering: When high-risk behavior is detected, the system triggers an alarm, which is expressed as: A= ; in, In alarm state, when When it belongs to the high-risk behavior set, an alarm is triggered; S73, automatic intervention measures: The system automatically takes intervention measures to stop the operation of dangerous equipment through linkage with the construction site machinery and equipment, expressed as: ; in, To stop the device, is the intervention function, based on the high-risk behavior set Determine the equipment that needs to be stopped; S74, linkage control: linkage between the system and the construction site machinery and equipment, and automatic stop of the equipment through control instructions, expressed as: ; in, is the device status, For the equipment that needs to be stopped, To stop the instruction; S75, emergency plan activation: After the alarm is triggered and automatic intervention measures are taken, the system activates the emergency plan and notifies relevant personnel to handle it, which is expressed as: ; in, It is an emergency plan state. A function to start the emergency plan.

[0041] By triggering alarms and automatically taking intervention measures, and coordinating with construction site machinery and equipment to stop the operation of dangerous equipment to prevent accidents from expanding, the response speed and processing efficiency of construction site safety management are improved. Automated control is also used to reduce the time for human intervention and improve the overall safety level.

[0042] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0043] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A construction site safety behavior recognition method based on deep learning, characterized in that: The following steps are involved: S1, data collection: collect environmental data of the construction site in real time through environmental perception sensors distributed throughout the construction site; S2, data preprocessing: multimodal fusion processing of different types of environmental data, association and integration, and preprocessing of the environmental data after multimodal fusion processing, including data cleaning, normalization and data enhancement; S3, feature extraction: use convolutional neural network to extract features from the preprocessed environmental data and extract feature vectors representing the behavior of construction site personnel; S4, individual behavior recognition and behavior correlation analysis: Use the extracted feature vectors to identify the specific behaviors of construction site personnel, including normal behaviors, abnormal behaviors, and potentially dangerous behaviors. Use biometric recognition technology to identify the identities of construction site personnel and correlate individual behaviors with their historical behavior data; S5, behavior prediction: combining historical behavior data and environmental change trends to predict human behavior and identify potential dangerous behaviors; S6, multi-level risk assessment: Based on the results of individual behavior identification and association analysis as well as the identification of potential dangerous behaviors, the identified behaviors and potential dangerous behaviors are subject to multi-level risk assessment according to risk levels; S7, emergency response and automatic intervention: After identifying high-risk behaviors, the system not only triggers an alarm, but also automatically takes intervention measures.

2. The construction site safety behavior identification method based on deep learning according to claim 1 is characterized in that: The S1 includes: S11, sensor placement: Environment perception sensors are placed in key areas of the construction site, including entrances, construction areas, danger zones, and rest areas. The number and placement of sensors are adjusted according to the scale and layout of the construction site. S12, camera data collection: using a camera to capture video data of the construction site in real time; S13, infrared data collection: using infrared sensors to collect thermal imaging data on the construction site; S14, temperature data collection: Use temperature sensors to collect ambient temperature data at various locations on the construction site in real time.

3. The construction site safety behavior identification method based on deep learning according to claim 1 is characterized in that: The S2 includes: S21, multimodal data fusion: different types of environmental data are synchronized in time and integrated into a unified data set. The weighted average method is used to perform multimodal fusion of different types of data, which is expressed as: ； in, For at time point The fusion characteristics of , and is the fusion weight coefficient, , and are the features of video, infrared and temperature data respectively; S22, data cleaning: filtering the fused data and removing the noise using the median filter algorithm; S23, normalization processing: normalizing different types of data; S24, data enhancement: enhance the data by random cropping, rotation, scaling and translation.

4. The construction site safety behavior identification method based on deep learning according to claim 1 is characterized in that: The S3 includes: S31, data input: input the preprocessed environmental data into the convolutional neural network. The input data is expressed as: ； in, is the input data set, For time point The fusion characteristics of S32, convolution layer: The input data is convolved through multiple convolution layers to extract low-level features. Each convolution layer uses multiple convolution kernels to perform feature mapping on the input data, which is expressed as: ； in, Output the pixel value in the feature map of the convolutional layer, is the pixel value of the input data, is the weight of the convolution kernel, is the bias term, is the spatial location of the input data, is the size of the convolution kernel, is the index of the convolution kernel; S33, Activation function: A non-linear activation function is applied after each convolutional layer. ReLU In order to introduce nonlinear characteristics, it can be expressed as: ； in, is the activation function output, is the output of the convolutional layer; Pooling layer: Max pooling is applied after the convolutional layer for dimensionality reduction and feature compression, expressed as: ； in, is the output of the pooling layer, is the input feature map, is the pooling step size, is the pooling window size; Fully connected layer: The features extracted by the convolution layer and the pooling layer are integrated and mapped to a high-dimensional feature space through the fully connected layer to generate a feature vector representing the behavior of the construction site personnel, which is expressed as: ； in, is the output feature vector, is the weight matrix of the fully connected layer, Output features of the pooling layer, is the bias term; Feature vector output: The final generated feature vector is expressed as: ； in, is the feature vector representing the behavior of construction site personnel, is the dimension of the feature vector.

5. The method for identifying construction site safety behaviors based on deep learning according to claim 1, characterized in that: The S4 includes: S41, behavior classification: the extracted feature vector is input into the pre-trained extreme gradient boosting model, and the extreme gradient boosting model processes the feature vector to identify the specific behavior of the construction site personnel, including normal behavior, abnormal behavior and potentially dangerous behavior; S42, biometric identification: Identify the identity of site personnel through biometric identification technology; S43, behavior association analysis: associate the identified individual behavior with its historical behavior data to establish a behavior pattern library; S44, behavior pattern update: based on the real-time monitored behavior data, the behavior pattern library is continuously updated and improved.

6. The method for identifying construction site safety behaviors based on deep learning according to claim 5, characterized in that: The S41 includes: S411, feature vector input: the preprocessed feature vector Input into the pre-trained extreme gradient boosting model; S412, Model training and optimization: Construction of tree structure: In each iteration, the extreme gradient boosting model updates the model by constructing a new decision tree to fit the residual of the current model, which is expressed as: ； in, For the The predicted value of a tree, Leaf Node The weight of is the indicator function, indicating the sample Is it a leaf node? ; S413, optimization objective function: The extreme gradient boosting model optimization objective function includes an error term and a regularization term, which is expressed as: ； in, is the objective function, is the loss function, is the true value, is the predicted value, For the The regularization term of the tree; S414, regularization term: controls the model complexity, expressed as: ； in, is the weight of the number of leaf nodes in the tree, is the number of leaf nodes, is the weight of the L2 regularization term, is the leaf node weight; S415, behavior classification: After each iteration, the extreme gradient boosting model accumulates the updated prediction results with the previous prediction results and finally predicts the specific behavior type, which is expressed as: ； in, For sample The final predicted value of For the The prediction results of the tree, is the number of iterations.

7. The method for identifying construction site safety behaviors based on deep learning according to claim 1, characterized in that: The S5 includes: S51, integration of historical behavior data and environmental data: historical behavior data and environmental data are time-aligned and integrated into a time series data set, expressed as: ； in, is a time series dataset, For time point behavioral characteristics, For time point Environmental characteristics; S52, feature extraction and preprocessing: extract and preprocess the integrated time series data to generate a high-dimensional feature vector that can reflect the characteristics of the time series, expressed as: ； S53, variational autoencoder training: train the feature vector using the variational autoencoder model; S54, Future Behavior Prediction: Predict the behavior characteristics at a future time point, expressed as: ； in, For the predicted future time point behavioral characteristics, is the current time point; S55, Dangerous Behavior Identification: Analyze the predicted behavioral characteristics to identify potential dangerous behaviors.

8. The method for identifying construction site safety behaviors based on deep learning according to claim 7, characterized in that: The S53 includes: S531, encoder: use the encoder to input feature vector Mapping to latent variables in latent space , generating the mean vector and the logarithmic variance vector , expressed as: ； in, is the encoder function, is the mean of the latent variable, is the log variance of the latent variable; S532, Reparameterization Technique: Use the reparameterization technique to sample latent variables from the latent space , expressed as: ； in, is the noise of standard normal distribution; S533, decoder: Use the decoder to convert the latent variables Mapping back to feature space, reconstructing the input feature vector , expressed as: ; in, is the decoder function, is the reconstructed feature vector; S534, loss function: The model is trained by optimizing the loss function of the variational autoencoder. The loss function includes reconstruction error and KL divergence, which is expressed as: ； in, is the reconstruction error, is the KL divergence, is the posterior distribution generated by the encoder, is the prior distribution.

9. The method for identifying construction site safety behaviors based on deep learning according to claim 1, characterized in that: The S6 includes: S61, integration of behavior identification and association analysis results: integrating the behaviors and potential dangerous behaviors identified in the individual behavior identification and association analysis steps; S62, integration of dangerous behavior identification results: integrating the potential dangerous behavior results identified in the behavior prediction step; S63, Risk level assessment: Based on the identified behaviors and potentially dangerous behaviors, assess the risk level of each behavior; S64, multi-level risk classification: the assessed risk levels are classified according to the preset risk level thresholds.

10. The method for identifying construction site safety behaviors based on deep learning according to claim 1, characterized in that: The S7 includes: S71, High-risk behavior identification: Identify high-risk behaviors in the multi-level risk assessment step; S72, alarm triggering: when high-risk behavior is detected, the system triggers an alarm; S73, automatic intervention measures: The system automatically takes intervention measures to stop the operation of dangerous equipment by linking with the construction site machinery and equipment; S74, linkage control: linkage between the system and the construction site machinery and equipment, realizing automatic stop of the equipment through control instructions; S75, emergency plan initiated: After the alarm is triggered and automatic intervention measures are taken, the system initiates the emergency plan and notifies relevant personnel to handle it.

Images