An AI-based method for detecting unsafe behaviors of construction workers

Abstract

This invention provides an artificial intelligence-based method for detecting unsafe behaviors of construction workers, belonging to the field of intelligent safety monitoring technology. This method addresses complex scenarios at construction sites, such as large variations in lighting, significant differences in target scale, and frequent personnel movement. It proposes a dual-stream fusion network. In the first stage, a multi-scale feature fusion network for small target detection is constructed. A new P2 high-resolution feature layer is added to YOLOv5, and a CoordConv coordinate channel and an occlusion recovery network (PORN) are introduced to improve detection accuracy for small targets such as those not wearing safety helmets or smoking, as well as in occluded scenarios. In the second stage, face-recognition is used to identify personnel within the behavior area. A feature temporal synchronization mechanism and a spatial alignment unit (SAU) are designed to automatically bind the behavior detection box and the face recognition box, effectively avoiding ID jumps. Finally, the method outputs the type of unsafe behavior, personnel identity, and temporal information, enabling automatic detection, accurate location, and traceable management of unsafe behaviors at construction sites.

Description

Technical Field

[0001] This invention relates to the field of intelligent safety monitoring technology, specifically to a method for detecting unsafe behaviors of construction workers based on artificial intelligence. Background Technology

[0002] Construction sites present complex working environments with frequent personnel movement, making manual supervision inefficient and prone to missed detections. Particularly in high-altitude work areas, behaviors such as not wearing safety helmets and smoking on-site are major causes of accidents. While existing deep learning-based target detection models, such as the YOLO series, are used for detecting safety helmets and reflective vests, the following prominent problems remain: extremely small targets like cigarettes are often missed or falsely detected due to their small size and complex backgrounds; because they are not linked to personnel identity, traditional detection systems can only identify behaviors but cannot determine the identity of the person performing the behavior, thus failing to form a traceable closed-loop management system; and the large variations in lighting and the numerous moving targets at construction sites result in insufficient model stability and inter-frame loss.

[0003] To address the shortcomings of existing technologies, this invention provides an AI-based method for detecting unsafe behaviors of construction workers. It employs a two-stage detection-recognition mechanism: the first stage utilizes Yolov5s for efficient unsafe behavior detection, and the second stage uses Face-recognition for identity verification. A feature temporal synchronization mechanism automatically binds detected unsafe behaviors to worker identities. To address issues such as significant variations in lighting and target scale at construction sites, a dynamic scene robustness enhancement module is introduced. This module improves video detection stability through operations such as optical flow-based inter-frame feature compensation, noise robustness enhancement training, and multi-domain style transfer data enhancement. For high-precision dynamic recognition of smoking at construction sites, a small target feature enhancement module is introduced to further enhance the model's ability to recognize small targets and occlusion situations. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an artificial intelligence-based method for detecting unsafe behaviors of construction workers, which solves the problem of insufficient scene adaptability of existing real-time monitoring algorithms for small targets and improves the detection accuracy of small targets in desert environments with large changes in light.

[0005] The technical solution adopted in this invention is to provide a method for detecting unsafe behaviors of construction workers based on artificial intelligence, including the following steps:

[0006] S1. Acquire the construction site monitoring video stream and output the data enhanced by the dynamic scene robustness enhancement module; S2. Input the enhanced data into the dual-stream fusion network. In the first stage, unsafe behavior detection is achieved based on the improved Yolov5s model. S3. The second stage is to achieve personnel identification based on the Face-recognition algorithm; S4. Achieve two-stage synchronous output through a feature-based temporal synchronization mechanism, outputting associated results and early warning information.

[0007] Preferably, in step S1, the optical flow estimation of the dynamic scene robustness enhancement module adopts the Farneback dense optical flow algorithm, which establishes motion relationships by calculating the pixel motion vectors of adjacent frames, fuses historical features with current frame features based on exponential moving average (EMA), compensates for feature loss caused by target motion, aligns the positions of the same object in adjacent frames through optical flow registration, and recovers some information of small targets at the feature layer.

[0008] Preferably, in step S1, the noise robustness training of the dynamic scene robustness enhancement module generates enhanced samples by adding additive Gaussian noise, Poisson noise, JPEG compression artifacts and simulated rain, snow and fog noise, and uses consistency loss constraint training. By introducing diverse noises during training and using consistency constraints, the network learns semantic representations that are insensitive to noise and retains the distinguishable features of small targets.

[0009] Preferably, in step S1, the multi-domain style transfer of the dynamic scene robustness enhancement module is implemented based on AdaIN statistical matching, which integrates content features and style features to generate domain-invariant features under different lighting and weather conditions, and finally integrates the three features through attention with adaptive weights.

[0010] Preferably, in step S2, the small target feature enhancement module adds a P2 feature layer to the Yolov5s Backbone, adds a high-resolution branch, adds coordinate channels before the convolutional layer through the CoordConv module, explicitly injects spatial location information, generates adaptive weights through a lightweight weight prediction subnetwork, and weightedly fuses the information from the P2, P3 and P4 feature layers.

[0011] Preferably, in step S2, the small target feature enhancement module includes an Occlusion Recovery Network (PORN), which adopts a lightweight U-Net architecture. It extracts multi-scale contextual information through an encoder, uses an attention mechanism to locate the occluded area, reconstructs the features of the occluded area through a decoder, and retains the original unoccluded features by combining residual connections, thus adapting to construction scenarios such as dust and safety helmet occlusion.

[0012] Preferably, step S2 also includes an improved Yolov5s model using an enhanced PANet structure, extending the P2 layer processing path, and achieving bottom-up and top-down feature fusion through a bidirectional feature pyramid. Each scale includes feature fusion and channel compression operations, adapting to special construction scenarios with large changes in light and high dust levels, such as desert photovoltaic projects.

[0013] Preferably, in step S3, the personnel identification method includes performing personnel identification on the detected unsafe behavior area, cropping the behavior area, detecting and aligning the face, generating facial features using a Face-recognition model and comparing them with a personnel database to determine the personnel identity.

[0014] Preferably, in step S4, the feature temporal synchronization mechanism includes a spatial alignment unit (SAU) that aligns the center of the face bounding box with the behavior space through linear mapping, thereby achieving accurate matching between the behavior detection box and the face detection box in the feature dimension.

[0015] Preferably, in step S4, the feature time-series synchronization mechanism includes spatiotemporal consistency analysis to realize the association between behavior and personnel by calculating a comprehensive score, and reduces ID jump problems by dual determination of timestamp synchronization, spatial IOU and center distance.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Significantly improves the accuracy of small object detection. By adding a P2 feature layer and a high-resolution branch to YOLOv5, the detection accuracy of cigarettes and safety helmets is improved. The CoordConv module explicitly injects coordinate information, enhancing the network's spatial awareness and solving the problem of spatial information loss in traditional convolution operations. The Occlusion Recovery Network (PORN) effectively handles common occlusion situations on construction sites, improving the recall rate for partially occluded small objects.

[0017] 2. The dynamic scene robustness enhancement module reduces target loss caused by camera shake / rapid movement through optical flow feature compensation, improves inter-frame tracking stability, and noise robustness training enables the model to maintain a recognition accuracy of over 85% under harsh conditions such as low light, high contrast and lens smudges.

[0018] 3. The feature-based temporal synchronization mechanism and spatial alignment unit (SAU) accurately associate behavior with the executor, reduce ID jump rate, and ensure that the detected unsafe behavior is accurately associated with the specific person through spatiotemporal consistency analysis.

[0019] 4. The optimized lightweight dual-stream fusion network structure reduces computing resource requirements and enables the adaptive weight allocation mechanism to dynamically adjust the fusion weights of different scale features in real time on edge devices, reducing redundant computing. The end-to-end integrated design reduces system complexity and single-point deployment costs, making it suitable for large-scale site deployment. Attached Figure Description

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is the overall flowchart of the system of the present invention; Figure 2This is a structural diagram of the dynamic scene robustness enhancement module of the present invention; Figure 3 This is a diagram of the YOLOv5 neural network structure of this invention. Detailed Implementation

[0021] To better understand the purpose, system architecture, and functional implementation of this embodiment, the embodiments and features described herein can be combined with each other without conflict. The exemplary embodiments disclosed herein will be described below with reference to the accompanying drawings, including specific technical details disclosed to aid understanding; however, these details should be considered exemplary rather than restrictive. Therefore, those skilled in the art should understand that various improvements and adjustments can be made to the embodiments described herein without departing from the scope and core ideas of the invention. Similarly, for clarity, detailed descriptions of well-known technologies, functions, and structures are omitted in the following description.

[0022] Example 1 Figure 1 This is the overall system flowchart of the present invention.

[0023] like Figure 1 As shown, the original video stream is obtained from the construction site monitoring camera, and the video frames are transmitted to the dynamic scene robustness enhancement module. The motion relationship between adjacent frames is estimated through the optical flow algorithm, and the features of the current frame are dynamically compensated to reduce the loss caused by the rapid movement of the target or the change of the viewpoint. The module simulates the noise such as low light, high contrast and lens dirt in the real construction site environment, enhances the adaptability of the model to harsh monitoring conditions, generates enhanced samples under different lighting and desert weather conditions, and expands the model's generalization ability.

[0024] The enhanced data is input into a dual-stream fusion network for processing. The first stage is unsafe behavior detection, which introduces an enhanced PANet optimized path aggregation network structure to strengthen the bidirectional fusion of shallow detail features and deep semantic features. An additional high-resolution feature path is added to handle small targets such as not wearing a helmet and smoking. CoordConv coordinate information is explicitly injected into the convolutional operation to enhance the network's spatial awareness. An Occlusion Recovery Network (PORN) is introduced to predict and recover occluded areas using contextual information, improving the recognition rate in complex scenes. Based on the enhanced features, high-risk unsafe behaviors such as not wearing a helmet and smoking on-site are detected.

[0025] The second stage is personnel identification. Image sub-regions are extracted from the unsafe behavior areas detected in the first-line detection. The face position is accurately located within the cropped area and standardized. Face detection and alignment are performed. The detected face is compared with the construction site personnel database through the Face-recognition algorithm to confirm the personnel identity. The two stages are output synchronously using the feature temporal synchronization mechanism. The behavior detection box and the face identification detection box are accurately aligned in the feature dimension using the Spatial Alignment Unit (SAU), as shown in the following formula (1).

[0026] (1) in, Center of the human face frame, For behavioral space, As a linear mapping, face bounding boxes and action bounding boxes are aligned to the same action space, ultimately outputting a fused and consistent feature pair.

[0027] By using a time synchronization mechanism to synchronize and bind the timestamps of behavior detection and face recognition, and through spatiotemporal consistency analysis, the detected unsafe behavior is accurately associated with the specific person performing the behavior, reducing the ID jump problem in the traditional two-stage solution, as shown in the following formula.

[0028] (2) in, To achieve a comprehensive consistency score considering time, space, and features, For behavior detection sequences, For face detection sequences, insecure behaviors are accurately bound to specific person IDs, eliminating ID jumps, misbinding, and multi-frame matching errors.

[0029] Finally, the system uses audible and visual alarms to prompt construction workers to correct unsafe behaviors in a timely manner, sends early warning information to automatically generate violation records, pushes them to on-site safety officers and management personnel, tracks the rectification progress, stores complete evidence of violations, clarifies the responsible party for violations, supports subsequent management decisions, and generates safety situation analysis based on historical data to assist in optimizing safety management.

[0030] Figure 2 This is a structural diagram of the dynamic scene robustness enhancement module of the present invention.

[0031] like Figure 2As shown, the dynamic scene robustness enhancement module is used to handle complex situations such as noise, lighting changes, shaking, and partial occlusion in construction site scenes. The input video frame eliminates the inconsistency in feature timing caused by camera shake / small movements through optical flow feature compensation, retains the spatiotemporal position information of small targets, estimates pixel-level optical flow for adjacent frames, fuses the features of the current frame with the registered features of adjacent frames to compensate for details, and uses exponential moving average (EMA) to stabilize historical features, as shown in the following formula.

[0032] (3) in, As a time-stable feature, As a historical feature, Using the exponent, optical flow registration aligns the positions of the same object in adjacent frames, thereby recovering / compensating for some information of small targets at the feature layer and improving the robustness of subsequent detection to small targets and jitter.

[0033] Noise robustness training improves the model's robustness to noisy environments such as camera noise, compression artifacts, rain, snow, and fog. Various noise transformations are applied to training frames to obtain noise-enhanced samples. Noise models include additive Gaussian, Poisson, multiplicative noise, JPEG compression artifacts, and synthetic rain / snow. These noise-enhanced samples are input into the network, and the feature extractor is trained using noise invariance constraints. Different noise versions of the same frame are expected to yield similar semantic features, which can be achieved through consistency loss. The reinforcement is shown in the following formula.

[0034] (4) in, Features of noise samples Using the original sample features, diverse noises are introduced during training and constrained by consistency. The network learns semantic representations that are insensitive to noise, while retaining the distinguishable features of small targets.

[0035] Style transfer enhances the model's generalization ability under different lighting, hue, and camera parameters, ensuring that features remain invariant across different style domains. Styles are explicitly injected or stripped using AdaIN's statistical matching formula, as shown in the following equation.

[0036] (5) Among them, Variance of style characteristics The variance of content features, The variance of style characteristics, The mean of the content features is used as the base value. The content features are then normalized using their own mean and variance to remove the original style. The mean and variance of the style features are then applied back to the normalized features, preserving the content structure. The style-aligned features are then output, resulting in domain-invariant feature output.

[0037] Finally, the time-stable, noise-robust, and domain-invariant features obtained from the three paths are combined into a stronger discriminative representation, which is then fused using attention with adaptive weights, as shown in the following equation.

[0038] (6) in, For time-stable fusion characteristics, Noise characteristics, For field-invariant features, This is a lightweight attention network. In the weighted fusion, a channel alignment function is used to ensure consistent channel count and learn the mapping. The output enhanced fusion features are then used by subsequent detection or recognition heads.

[0039] Figure 3 This is a diagram of the YOLOv5 neural network structure of this invention.

[0040] like Figure 3 As shown, the backbone network maintains the original YOLOv5 CSPDarknet53 backbone network structure. P2 (160×160), P3 (80×80), and P4 (40×40) feature maps are extracted from the intermediate layers. The P2 feature layer is added as a high-resolution feature source, specifically for small object detection. The CoordConv module is introduced, which explicitly adds two coordinate channels (x, y) before the standard convolutional layer. The coordinate channel values ​​are normalized to [-1, 1], making the network more sensitive to changes in spatial position, preserving the accurate spatial information of small objects, and solving the problem of spatial position information loss in traditional convolution operations.

[0041] The enhanced PANet feature pyramid extends the traditional PANet structure by adding a P2 layer processing path to achieve three-scale feature fusion. It employs a bidirectional feature pyramid structure, combining bottom-up and top-down paths, with each scale including feature fusion and channel compression operations. The cross-scale feature fusion module utilizes an adaptive weight allocation mechanism to calculate adaptive weights for the P2, P3, and P4 feature layers. A lightweight weight prediction subnetwork dynamically adjusts the weights generated using global average pooling, fully connected layers, and sigmoid activation based on the feature content. Multi-scale feature alignment upsamples the P3 and P4 features to P2 resolution while preserving spatial detail, and weighted fusion dynamically adjusts the features at different scales.

[0042] The occlusion recovery network utilizes the U-net encoder to extract multi-scale contextual information and capture long-range dependencies. An attention mechanism module focuses on the relevant context surrounding the occluded area, and then a feature decoder reconstructs the features of the occluded area. Residual connections preserve information from the original unoccluded portion, and a self-supervised learning method learns occlusion patterns to adapt to different types of occlusion, such as dust, hard hat shadows, and tool occlusion. It is optimized for small target detection in complex construction site environments, particularly suitable for scenarios with large lighting variations and high dust levels, such as desert environments, significantly improving the detection accuracy of critical small targets such as seat belts and cigarettes.

[0043] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0044] The specific embodiments described above do not constitute a limitation on the scope of protection disclosed in this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for detecting unsafe behaviors of construction workers based on artificial intelligence, characterized in that, Includes the following steps: S1. Acquire the construction site monitoring video stream and output the data enhanced by the dynamic scene robustness enhancement module; S2. Input the enhanced data into the dual-stream fusion network. In the first stage, unsafe behavior detection is achieved based on the improved Yolov5s model. S3. The second stage is to achieve personnel identification based on the Face-recognition algorithm; S4. Achieve two-stage synchronous output through a feature-based temporal synchronization mechanism, outputting associated results and early warning information.

2. The method according to claim 1, characterized in that, step In S1, the optical flow estimation of the dynamic scene robustness enhancement module adopts the Farneback dense optical flow algorithm, which establishes motion relationships by calculating the motion vectors of pixels in adjacent frames, and fuses historical features with current frame features based on exponential moving average (EMA). To compensate for feature loss caused by target motion, optical flow registration is used to align the positions of the same object in adjacent frames, and partial information of small targets is recovered at the feature layer.

3. The method according to claim 1, characterized in that, step In S1, the noise robustness training of the dynamic scene robustness enhancement module generates enhanced samples by adding additive Gaussian noise, Poisson noise, JPEG compression artifacts, and simulated noise from rain, snow, and fog. It adopts consistency loss constraint training. By introducing diverse noises during training and using consistency constraints, the network learns semantic representations that are insensitive to noise and retains the distinguishable features of small targets.

4. The method according to claim 1, characterized in that, step In S1, the multi-domain style transfer of the dynamic scene robustness enhancement module is implemented based on AdaIN statistical matching, which integrates content features and style features to generate domain-invariant features under different lighting and weather conditions, and finally integrates the three features through attention with adaptive weights.

5. The method according to claim 1, characterized in that, In step S2, the small target feature enhancement module adds a P2 feature layer to the Yolov5s Backbone, adds a high-resolution branch, adds coordinate channels before the convolutional layer through the CoordConv module, explicitly injects spatial location information, generates adaptive weights through a lightweight weight prediction subnetwork, and weightedly fuses the information from the P2, P3 and P4 feature layers.

6. The method according to claim 1, characterized in that, In step S2, the small target feature enhancement module includes an occlusion recovery network (PORN), which adopts a lightweight U-Net architecture. The encoder extracts multi-scale contextual information, uses an attention mechanism to locate the occluded area, the decoder reconstructs the features of the occluded area, and combines residual connections to retain the original unoccluded features, adapting to construction scenarios such as dust and safety helmet occlusion.

7. The method according to claim 1, characterized in that, Step S2 also includes an improved Yolov5s model that adopts an enhanced PANet structure, expands the P2 layer processing path, and achieves bottom-up and top-down feature fusion through a bidirectional feature pyramid. Each scale includes feature fusion and channel compression operations, which are adapted to special construction scenarios with large changes in light and a lot of dust, such as desert photovoltaic projects.

8. The method according to claim 1, characterized in that, In step S3, the personnel identification method includes performing personnel identification on the detected unsafe behavior area, cropping the behavior area, detecting and aligning the face, generating facial features using the Face-recognition model and comparing them with the personnel database to determine the personnel identity.

9. The method according to claim 1, characterized in that, In step S4, the feature temporal synchronization mechanism includes a spatial alignment unit (SAU) that aligns the center of the face bounding box with the behavior space through linear mapping, thereby achieving accurate matching between the behavior detection box and the face detection box in the feature dimension.

10. The method according to claim 1, characterized in that, In step S4, the feature time-series synchronization mechanism includes spatiotemporal consistency analysis to realize the association between behavior and personnel by calculating a comprehensive score, and to reduce ID jump problems through dual judgment of timestamp synchronization, spatial IOU and center distance.

Images