Construction site personnel unsafe behavior detection method and system based on GAN
Through the unsupervised learning method based on GAN, a generative adversarial network model framework is designed, which solves the problem of inefficiency in traditional safety supervision, realizes intelligent detection of unsafe behavior of construction site personnel, and improves the level of safe production.
Patent Information
- Application Number
- CN202411455998.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-02-14
AI Technical Summary
The traditional safety supervision method of construction site is inefficient and cannot effectively monitor complex and dynamic construction environments. The existing deep learning methods rely on a large amount of labeled data, making it difficult to obtain prior knowledge of the unsafe behavior of construction site personnel.
Adopting an unsupervised learning method based on GAN, a generative adversarial network model framework is designed, and through the adversarial training of generators and discriminators, the normal and unsafe behavior of construction site personnel is learned to realize intelligent detection of unsafe behavior.
It has realized intelligent detection of unsafe behavior of construction site personnel, improved the level of production safety, improved management capabilities, and provided new ideas and methods for preventing production safety accidents.
Smart Images
Figure CN119445471A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of construction site safety technology, and more specifically, to a GAN-based construction site personnel unsafe behavior detection method and system. Background Art
[0002] Safety management of construction sites has always been a difficult issue that the industry is concerned about. In the context of construction safety, more than 80% of accidents on construction sites are the result of unsafe behaviors of workers. The causes of unsafe behaviors of people mainly depend on two factors: one is the safety awareness and attitude of workers towards construction risks, which mainly depends on safety education and intervention for construction workers, and safety education is difficult to cover comprehensively and fully in most cases; the other is safety supervision. Discovering unsafe behaviors of construction workers at the first time and taking timely measures will greatly reduce casualties, but traditional enterprises and regulatory departments still rely on dedicated personnel to supervise the safety behaviors of construction workers, which has low supervision efficiency and poor timeliness.
[0003] However, traditional safety supervision methods are inefficient and cannot effectively monitor complex and dynamic construction environments. In the past decade, deep learning technology has developed rapidly and has been widely used to improve the safety of construction sites. Deep learning and computer vision technologies are widely used in the detection and early warning of unsafe behaviors of construction site personnel. Related research mainly focuses on aspects such as personal protective equipment detection. Most of these methods are based on supervised or semi-supervised learning and require a large number of professionally annotated image samples for model training. However, the prior knowledge of unsafe behaviors of construction site personnel is usually very limited, and it is also difficult to obtain relevant behavioral data sets. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, the present invention provides a GAN-based method and system for detecting unsafe behaviors of construction site personnel, which has the advantage of more precise operation.
[0005] To achieve the above object, the present invention provides the following technical solution: a method for detecting unsafe behaviors of construction site personnel based on GAN, comprising the following steps:
[0006] Step 1: Conduct literature analysis and summary, and select representative application scenarios of unsafe behaviors of personnel;
[0007] Step 2: Collect and construct a video dataset of construction site personnel behavior containing normal and unsafe behaviors;
[0008] Step 3: Design a generative adversarial network model framework for unsupervised learning;
[0009] Step 4: Train the proposed model on a self-built dataset, evaluate the model performance and compare it with the baseline model;
[0010] Step 5: Eliminate each pseudo-anomaly generation module through ablation experiments and evaluate its contribution to the model performance.
[0011] As a preferred technical solution of the present invention, in step one, the process of literature analysis and summary is to summarize the current research status at home and abroad to conduct accident statistical analysis, case analysis, and literature analysis.
[0012] As a preferred technical solution of the present invention, in step 2, the data set is the WeTeam22 data set, which collects the postures and directions of normal and unsafe behaviors of construction workers on the edge of a tower crane at a construction site, wherein normal behaviors include walking and standing, and unsafe behaviors include throwing objects, climbing over safety guardrails, sitting and leaning on safety guardrails, and standing and leaning on safety guardrails.
[0013] As a preferred technical solution of the present invention, the specific process of step 2 is:
[0014] The background scene of the tower crane at the construction site is determined to construct an image dataset containing normal and unsafe behaviors;
[0015] and develop data collection, processing and selection plans to ensure data set quality;
[0016] Data collection, cleaning, processing and selection of data subsets for model experiments were performed to construct the WeTeam22 dataset that includes both single-person and multi-person behaviors.
[0017] As a preferred technical solution of the present invention, in the step three, the unsupervised learning generative adversarial network model framework consists of a generator, a discriminator and five pseudo-anomaly generation modules; wherein the generator uses an encoder-decoder architecture, including an encoder network and a decoder network; the discriminator uses ResNet-18, and then adds a fully connected layer and a sigmoid activation function; the five pseudo-anomaly generation modules are respectively time pseudo-anomaly generation modules A and B, OGNet pseudo-anomaly generation module C, fusion channel pseudo-anomaly generation module D, and M IX frame fusion pseudo-anomaly generation module E.
[0018] As an optimal technical solution of the present invention, the encoder network is composed of four layers of three-dimensional convolutional layers connected in sequence, each layer of convolution kernel size is 3×3×3, and a downsampling operation with a step size of 2 is adopted in the spatiotemporal domain to reduce the spatiotemporal resolution and the number of channels in stages, and finally obtain a latent space representation; the decoder network adopts a symmetrical four-layer three-dimensional deconvolution layer upsampling structure, each layer of deconvolution kernel size is 3×3×3, and the spatiotemporal resolution and the number of channels of the input sequence are gradually restored by adjusting the step size and output padding parameters; the output channel number of the last deconvolution layer is set to 1, and a tanh activation function is connected to output a reconstruction result in the range of [-1,1]; the discriminator is built on the ResNet-18 model, and the output number of the ResNet-18 fully connected layer is modified to 1; a Sigmad activation layer is added after the FC layer to map the output value to between 0 and 1, indicating the probability of the input image being of good reconstruction quality and poor quality.
[0019] As a preferred technical solution of the present invention, in step 4, the training of the model is divided into two stages; the first stage is the pre-training of the generator, and the second stage is adversarial training;
[0020] In the first stage, the generator’s autoencoder is trained to improve the generator’s ability to reconstruct normal sequences; an early version of the generator from the first stage is saved to generate reconstruction results of lower quality;
[0021] In the second stage, the discriminator is updated using an earlier version of the generator trained in the first stage; the high-quality examples input to the discriminator come from real normal video frames and their reconstruction results generated by the generator, and the low-quality examples come from five pseudo-anomaly generation modules.
[0022] As a preferred technical solution of the present invention, in the step 4, the model performance evaluation uses the area under the frame-level curve as an evaluation indicator, and the baseline models come from the OGNet baseline method and the STEAL Net baseline method respectively.
[0023] As a preferred technical solution of the present invention, in step five, the ablation experiment evaluates the effect of different pseudo-anomaly generation modules on model performance by comparing the AUC values under different combinations of pseudo-anomaly generation modules; by fixing one of the pseudo-anomaly generation modules and changing the combination of other modules to analyze the relative contribution of different modules to performance improvement, the influence of different pseudo-anomaly generation modules on improving model performance is evaluated.
[0024] As a preferred technical solution of the present invention, the present invention also provides a GAN-based construction site personnel unsafe behavior detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, it implements the steps of the above-mentioned GAN-based construction site personnel unsafe behavior detection method.
[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention can realize intelligent detection of unsafe behaviors of workers by using normal behavior data training, and improves the level of safe production by using information technology for safety management, and enhances the management ability of construction personnel for unsafe behaviors, providing new ideas and methods for preventing safe production accidents. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is a schematic diagram of the structure of the present invention;
[0027] Figure 2 This is a specific flow chart of step 2;
[0028] Figure 3 Examples of normal and unsafe behaviors are shown in the figure;
[0029] Figure 4 This is the schematic diagram of the framework structure of the adversarial network model;
[0030] Figure 5 Schematic diagram of the encoder-decoder structure. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] like Figures 1 to 5 As shown, any one embodiment of the present invention is described for further explanation:
[0033] A GAN-based method for detecting unsafe behaviors of construction site personnel includes the following steps:
[0034] Step 1: Conduct literature analysis and summary, and select representative application scenarios of unsafe behaviors of personnel.
[0035] First, we summarize the current status of research at home and abroad, conduct accident statistics analysis, case analysis, and literature analysis, and select representative application scenarios of unsafe behaviors of personnel.
[0036] Step 2: Collect and construct a video dataset of construction site personnel behavior containing normal and unsafe behaviors.
[0037] The dataset is the WeTeam22 dataset, which collects the postures and orientations of normal and unsafe behaviors of construction workers near tower cranes at construction sites. Normal behaviors include walking and standing, and unsafe behaviors include throwing objects, climbing over safety guardrails, sitting and leaning on safety guardrails, and standing and leaning on safety guardrails.
[0038] The specific process is:
[0039] The background scene of the tower crane at the construction site is determined to construct an image dataset containing normal and unsafe behaviors;
[0040] and develop data collection, processing and selection plans to ensure data set quality;
[0041] Data collection, cleaning, processing and selection of data subsets for model experiments were performed to construct the WeTeam22 dataset that includes both single-person and multi-person behaviors.
[0042] Step 3: Design a generative adversarial network model framework for unsupervised learning.
[0043] The generative adversarial network model framework for unsupervised learning consists of a generator, a discriminator and five pseudo-anomaly generation modules; the generator uses an encoder-decoder architecture, including an encoder network and a decoder network; the discriminator uses ResNet-18, and then adds a fully connected layer and a sigmoid activation function; the five pseudo-anomaly generation modules are temporal pseudo-anomaly generation modules A and B, OGNet pseudo-anomaly generation module C, fusion channel pseudo-anomaly generation module D, and MIX frame fusion pseudo-anomaly generation module E. The encoder network consists of four layers of three-dimensional convolutional layers connected in sequence, each layer of convolution kernel size is 3×3×3, and a downsampling operation with a step size of 2 is used in the spatiotemporal domain to reduce the spatiotemporal resolution and the number of channels in stages, and finally obtain the latent space representation; the decoder network adopts a symmetrical four-layer three-dimensional deconvolution layer upsampling structure, each layer of deconvolution kernel size is 3×3×3, and the spatiotemporal resolution and the number of channels of the input sequence are gradually restored by adjusting the step size and output padding parameters; the number of output channels of the last deconvolution layer is set to 1, and a tanh activation function is connected to output the reconstruction result in the range of [-1,1]; the discriminator is built on the ResNet-18 model, and the output number of the ResNet-18 fully connected layer is modified to 1; a Sigmoid activation layer is added after the FC layer to map the output value to between 0 and 1, indicating the probability of the input image being of good and poor reconstruction quality.
[0044] Pseudo-anomaly generation module A: jumps the middle frame of the sequence generated by the generator of the s-frame sequence.
[0045] Pseudo-anomaly generation module B: An example of the fusion of the first and last frames of a skipped frame sequence after reconstruction by the generator.
[0046] Pseudo-anomaly generation module C: Examples reconstructed by the generator with low training epochs.
[0047] Pseudo-anomaly generation module D: An example of three-frame channel fusion after the normal frame sequence is reconstructed by the generator.
[0048] Pseudo-anomaly generation module E: An example of the fusion of the first frame and the last frame of a normal frame sequence after reconstruction by the generator.
[0049] Normal images and normal image reconstructions are used as positive samples, and images generated by the five pseudo-abnormal generation modules are used as negative samples as input to the discriminator. The generator and the discriminator learn a class of data representations in an unsupervised adversarial manner. The generator attempts to continuously update its ability to reconstruct normal behavior images. At the same time, the discriminator continuously optimizes its ability to distinguish between good and bad image reconstruction quality. Finally, the discriminator trained by GAN is used to perform abnormality scoring.
[0050] Step 4: Train the proposed model on a self-built dataset, evaluate the model performance and compare it with the baseline model;
[0051] In the first stage, a low-epoch generator is trained and saved for the second stage of training. In the second stage, the discriminator and generator are jointly trained so that the generator can generate more realistic positive samples and the discriminator can better judge the reconstruction quality of the input image. The trained discriminator is used to judge whether the input video frame is abnormal.
[0052] The essence of the first stage of training is to obtain a generative model with basic reconstruction capabilities.
[0053] The essence of the second stage of training is to provide the discriminator with reconstruction examples of good and poor quality, in order to let it understand what kind of output the generator will produce in the case of abnormal input. By letting the discriminator learn to distinguish various artificially constructed fake samples, the discriminant boundary can be quickly improved, and the training is only carried out for a few epochs because the discriminator will converge quickly.
[0054] Specifically, as the input of the discriminator, examples with good quality come from real data and data reconstructed by the generator. Examples with poor quality come from pseudo-anomaly generation modules A to E.
[0055] The distorted examples generated by the pseudo-anomaly generation module are perturbations of the true data distribution and can be viewed as reconstructed anomalous samples from the perspective of the generator, which can perform high-quality reconstruction. Therefore, without explicitly using any out-of-class data, the behavior of the generator on outlier data is indirectly imitated to train the discriminator. The discriminator directly learns the behavior of the generator in the presence of anomalous or abnormal inputs. This can explicitly help the discriminator learn to reconstruct the underlying patterns of anomalous data, resulting in a more robust anomaly detection model. Therefore, by redefining the learning paradigm of the discriminator to be consistent with the traditional generative single-class learning model, that is, the reconstruction quality of data from known classes is better than that of data from unknown or abnormal classes.
[0056] The model performance evaluation uses the frame-level area under the curve as the evaluation indicator, and the baseline models come from the OGNet baseline method and the STEAL Net baseline method respectively.
[0057] Step 5: Eliminate each pseudo-anomaly generation module through ablation experiments and evaluate its contribution to the model performance.
[0058] The ablation experiment evaluates the effect of different pseudo-anomaly generation modules on model performance by comparing the AUC values under different combinations of pseudo-anomaly generation modules. The ablation experiment also analyzes the relative contribution of different modules to performance improvement by fixing one of the pseudo-anomaly generation modules and changing the combination of other modules. It also evaluates the impact of different pseudo-anomaly generation modules on improving model performance.
[0059] The present invention also discloses a GAN-based unsafe behavior detection system for construction site personnel, comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of a GAN-based unsafe behavior detection method for construction site personnel are implemented as described in any one of claims 1 to 9.
[0060] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0061] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A GAN-based method for detecting unsafe behaviors of construction site personnel, characterized in that: The steps include: Step 1: Conduct literature analysis and summary, and select representative application scenarios of unsafe behaviors of personnel; Step 2: Collect and construct a video dataset of construction site personnel behavior containing normal and unsafe behaviors; Step 3: Design a generative adversarial network model framework for unsupervised learning; Step 4: Train the proposed model on a self-built dataset, evaluate the model performance and compare it with the baseline model; Step 5: Eliminate each pseudo-anomaly generation module through ablation experiments and evaluate its contribution to the model performance.
2. According to claim 1, a GAN-based construction site personnel unsafe behavior detection method is characterized by: In step one, the process of literature analysis and summary is to summarize the current research status at home and abroad and conduct accident statistical analysis, case analysis, and literature analysis.
3. The method for detecting unsafe behaviors of construction site personnel based on GAN according to claim 1, characterized in that: In step 2, the data set is the WeTeam22 data set, which collects the postures and directions of normal and unsafe behaviors of construction workers near tower cranes at construction sites, where normal behaviors include walking and standing, and unsafe behaviors include throwing objects, climbing over safety guardrails, sitting and leaning on safety guardrails, and standing and leaning on safety guardrails.
4. The method for detecting unsafe behaviors of construction site personnel based on GAN according to claim 1, characterized in that: The specific process of step 2 is as follows: The background scene of the tower crane at the construction site is determined to construct an image dataset containing normal and unsafe behaviors; and develop data collection, processing and selection plans to ensure data set quality; Data collection, cleaning, processing and selection of data subsets for model experiments were performed to construct the WeTeam22 dataset that includes both single-person and multi-person behaviors.
5. The method for detecting unsafe behaviors of construction site personnel based on GAN according to claim 1, characterized in that: In the step three, the unsupervised learning generative adversarial network model framework consists of a generator, a discriminator and five pseudo-anomaly generation modules; wherein the generator uses an encoder-decoder architecture, including an encoder network and a decoder network; the discriminator uses ResNet-18, and then adds a fully connected layer and a sigmoid activation function; the five pseudo-anomaly generation modules are respectively time pseudo-anomaly generation modules A and B, OGNet pseudo-anomaly generation module C, fusion channel pseudo-anomaly generation module D, and MIX frame fusion pseudo-anomaly generation module E.
6. The method for detecting unsafe behaviors of construction site personnel based on GAN according to claim 5, characterized in that: The encoder network is composed of four layers of three-dimensional convolutional layers connected in sequence, each layer of convolution kernel size is 3×3×3, and a downsampling operation with a step size of 2 is used in the spatiotemporal domain to reduce the spatiotemporal resolution and the number of channels in stages, and finally obtain a latent space representation; the decoder network adopts a symmetrical four-layer three-dimensional deconvolution layer upsampling structure, each layer of deconvolution kernel size is 3×3×3, and the spatiotemporal resolution and the number of channels of the input sequence are gradually restored by adjusting the step size and output padding parameters; the number of output channels of the last deconvolution layer is set to 1, and a tanh activation function is connected to output a reconstruction result in the range of [-1,1]; the discriminator is built on the ResNet-18 model, and the output number of the ResNet-18 fully connected layer is modified to 1; A Sigmoid activation layer is added after the FC layer to map the output value to between 0 and 1, indicating the probability of the input image being of good or poor reconstruction quality.
7. The method for detecting unsafe behaviors of construction site personnel based on GAN according to claim 1, characterized in that: In step 4, the training of the model is divided into two stages; the first stage is the pre-training of the generator, and the second stage is adversarial training; In the first stage, the generator’s autoencoder is trained to improve the generator’s ability to reconstruct normal sequences; an early version of the generator from the first stage is saved to generate reconstruction results of lower quality; In the second stage, the discriminator is updated using the earlier version of the generator trained in the first stage; The high-quality examples input to the discriminator come from real normal video frames and their reconstruction results generated by the generator, while the low-quality examples come from five pseudo-anomaly generation modules.
8. The method for detecting unsafe behaviors of construction site personnel based on GAN according to claim 1, characterized in that: In the step 4, the model performance evaluation uses the area under the frame-level curve as an evaluation indicator, and the baseline models come from the OGNet baseline method and the STEAL Net baseline method respectively.
9. The method for detecting unsafe behaviors of construction site personnel based on GAN according to claim 1, characterized in that: In the step five, the ablation experiment evaluates the effect of different pseudo-anomaly generation modules on the model performance by comparing the AUC values under different combinations of pseudo-anomaly generation modules; by fixing one of the pseudo-anomaly generation modules and changing the combination of other modules, the relative contribution of different modules to the performance improvement is analyzed, and the effect of different pseudo-anomaly generation modules on improving the model performance is evaluated.
10. A GAN-based construction site personnel unsafe behavior detection system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of a method for detecting unsafe behavior of construction site personnel based on GAN as described in any one of claims 1-9 are implemented.