Construction personnel unsafe behavior detection method based on improved YOLO

Abstract

The invention discloses a construction personnel unsafe behavior detection method based on an improved YOLO, and relates to the technical field of behavior detection, and the method comprises the steps: carrying out the improvement and training of the YOLO through the marked video data, obtaining an improved YOLO model, obtaining the video data of the operation motion of a to-be-recognized construction personnel, inputting the video data into the improved YOLO model, and combining with a multi-target tracking algorithm, and carrying out the recognition of the operation motion of the to-be-recognized construction personnel. Performing quantitative analysis on the behavior types and the behavior track characteristics of the construction personnel by applying a quantitative theory, and performing evaluation in combination with a safety evaluation model constructed based on a fuzzy analytic hierarchy process to obtain a comprehensive safety score of the construction personnel. And generating an unsafe behavior detection report according to the comprehensive safety score and the behavior safety level of the construction personnel. Through the YOLO behavior detection method, the accuracy and efficiency of unsafe behavior detection of the construction personnel are greatly improved.

Description

[0001] The present application relates to the technical field of behavior detection, and in particular to a construction worker unsafe behavior detection method based on improved YOLO. BACKGROUND

[0002] With the development of computer vision technology, especially the breakthrough progress in the field of deep learning, target detection methods have been widely applied in various industries. As one of the labor-intensive industries, the construction industry has higher requirements for the safety management of construction workers due to the complex construction environment and high risk coefficient. In recent years, target detection algorithms have gradually become a research hotspot, and the YOLO series model has attracted much attention due to its high efficiency and accuracy. YOLO realizes the rapid positioning and identification of targets by dividing an image into grids and predicting the bounding box and class probability in each grid, which provides an implementable method path for the automated monitoring of unsafe behaviors in construction sites.

[0003] Although the existing YOLO performs well in the task of target detection in general scenarios, there is still room for improvement in the detection of construction worker unsafe behaviors in specific application scenarios such as construction sites. First, the complex lighting conditions, variable weather conditions, and dynamically changing work environment of the construction site pose challenges to the robustness of the model. Existing YOLO often cannot effectively cope with interference factors, resulting in high false detection rate and missed detection rate. In addition, most existing methods only stay at the behavior recognition level and lack in-depth analysis of the specific behavior trajectory of construction workers, which cannot comprehensively evaluate the behavior risk level. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a construction worker unsafe behavior detection method based on improved YOLO to solve the problems of false detection, missed detection, and in-depth analysis of behavior trajectory of construction workers.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a construction worker unsafe behavior detection method based on improved YOLO, which includes,

[0008] Collecting labeled video data, the labeled video data including action videos of construction workers during construction operations;

[0009] Improving and training YOLO using labeled video data to obtain an improved YOLO model;

[0010] The video data input of the to-be-identified construction personnel operation action is acquired, an improved YOLO model is used, and a multi-target tracking algorithm is combined to obtain behavior category and behavior trajectory features of the construction personnel;

[0011] The behavior category and behavior trajectory features of the construction personnel are subjected to quantitative analysis by using a quantification theory, and a safety evaluation model based on a fuzzy analytic hierarchy process is combined to perform evaluation, so as to obtain a comprehensive safety score of the construction personnel;

[0012] According to the comprehensive safety score and the behavior safety level of the construction personnel, an unsafe behavior detection report is generated.

[0013] As a preferred scheme of the construction personnel unsafe behavior detection method based on the improved YOLO, the collection of the labeled video data is that the action video of the construction personnel during construction operation is labeled by using a labeling tool LabelImg, and the labeled video is disassembled into independent images by using a tool ffmpeg.

[0014] As a preferred scheme of the construction personnel unsafe behavior detection method based on the improved YOLO, the improved YOLO model is obtained by using a YOLO architecture as a basic target detection model, adding a attention enhancement mechanism and a multi-scale fusion mechanism in the YOLO model, and training and verifying the improved YOLO model by using the labeled video data.

[0015] As a preferred scheme of the construction personnel unsafe behavior detection method based on the improved YOLO, the adding of the attention enhancement mechanism and the multi-scale fusion mechanism is specifically,

[0016] Global average pooling is performed on each channel of the YOLO model to generate a channel descriptor, the importance weight of each channel is calculated by using the full connection layer of the YOLO model and the channel descriptor, and each channel of the image in the YOLO model is weighted by using the importance weight to obtain an attention weight map;

[0017] Max-pooling and average-pooling are respectively performed on the feature maps in the YOLO model along the channel to obtain two two-dimensional spatial descriptors, the two descriptors are combined by using the convolution layer of the YOLO model to generate a spatial attention map, and the addition of the attention enhancement mechanism is completed;

[0018] A BiFPN unit is constructed in the feature pyramid structure of the YOLO model, and the information interaction and weighted fusion between the feature maps at different levels in the YOLO model are performed by using the BiFPN unit, and the addition of the multi-scale fusion mechanism in the YOLO model is completed.

[0019] As a preferred scheme of the construction personnel unsafe behavior detection method based on improved YOLO, wherein: the video data of the to-be-recognized construction personnel operation action is input into the improved YOLO model, and a multi-target tracking algorithm is combined to obtain the behavior category and behavior trajectory of the construction personnel, specifically,

[0020] Real-time collection of video streams of activities of each construction personnel in the to-be-tested construction site;

[0021] After the video stream is unified as the input requirement of the YOLO model, the video stream is input into the improved YOLO model to obtain the behavior category of the construction personnel;

[0022] The average moving speed, trajectory curvature and stay time of the construction personnel in the construction area in the video stream are extracted using the DeepSORT multi-target tracking algorithm to obtain the behavior trajectory feature of the construction personnel.

[0023] As a preferred scheme of the construction personnel unsafe behavior detection method based on improved YOLO, wherein: the behavior category and behavior trajectory feature of the construction personnel are quantitatively analyzed by using the quantification theory, specifically,

[0024] A behavior risk level mapping table is established, the behavior category of the construction personnel is mapped into the risk level mapping table, and a corresponding risk value is assigned;

[0025] The occurrence frequency of the behavior category of the construction personnel is counted, and the behavior risk score of the construction personnel is calculated by combining the risk value of the behavior category;

[0026] The sliding window is used to extract statistical indicators for the behavior trajectory feature of the construction personnel, and the linear weighted summation is combined to calculate the behavior trajectory score of the construction personnel.

[0027] As a preferred scheme of the construction personnel unsafe behavior detection method based on improved YOLO, wherein: the safety evaluation model constructed based on the fuzzy analytic hierarchy process is combined for evaluation to obtain the comprehensive safety score of the construction personnel operation action, specifically,

[0028] The fuzzy analytic hierarchy process is used to construct the target layer, the criterion layer and the index layer in sequence to obtain the safety evaluation model;

[0029] The behavior risk score and the behavior trajectory score of the construction personnel are input into the safety evaluation model for weight distribution and consistency check to obtain the weight of the behavior risk and the behavior trajectory;

[0030] Based on the behavior risk score, the behavior trajectory score and the corresponding weight of the construction personnel, the fusion calculation is performed by combining the time decay factor to obtain the comprehensive safety score of the construction personnel;

[0031] The preset behavior safety level interval maps the comprehensive safety score of the construction personnel to the preset behavior safety level interval, to obtain a behavior safety level of the construction personnel.

[0032] As a preferred scheme of the construction personnel unsafe behavior detection method based on improved YOLO, wherein: the unsafe behavior detection report is generated according to the comprehensive safety score and the behavior safety level of the construction personnel, specifically,

[0033] The comprehensive safety score and the behavior safety level of the construction personnel are input into a report generation unit, to obtain a construction personnel behavior event list and an occurrence time;

[0034] The behavior event list and the occurrence time information are input into a structured template engine, to generate the construction personnel unsafe behavior detection report.

[0035] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, any step of the construction personnel unsafe behavior detection method based on improved YOLO is realized.

[0036] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by the processor, any step of the construction personnel unsafe behavior detection method based on improved YOLO is realized.

[0037] The present application has the following beneficial effects: through data collection, model improvement, behavior recognition and evaluation, a comprehensive and efficient construction personnel unsafe behavior detection mechanism is constructed, first, the YOLOv3 model is deeply improved to improve the accuracy of the YOLOv3 model in the complex construction site environment, second, combined with an advanced multi-target tracking algorithm, the behavior trajectory of the construction personnel is monitored and recorded in real time, not only various behavior actions can be accurately recognized, but also potential safety hazards can be effectively captured, the behavior risk is quantitatively evaluated by using the quantification theory and the fuzzy analytic hierarchy process, the complex on-site behavior is converted into a specific risk score, the scientificity and operability of safety management are improved, a detailed unsafe behavior detection report is generated, the overall construction personnel unsafe behavior detection accuracy and efficiency are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0039] Fig. 1 Flow chart of the improved YOLO-based construction worker unsafe behavior detection method.

[0040] Fig. 2 Schematic diagram of the improved YOLOv3 model.

[0041] Fig. 3 Flow chart of performing fuzzy analytic hierarchy process.

[0042] Fig. 4 Flow chart of construction worker behavior detection and tracking. DETAILED DESCRIPTION

[0043] In order to make the above objectives, features and advantages of the present application more apparent and comprehensible, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0044] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be practiced in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the concept of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0045] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0046] Reference Figs. 1-4 For one embodiment of the present application, the embodiment provides an improved YOLO-based construction worker unsafe behavior detection method, comprising the following steps:

[0047] S1, collect labeled video data, specifically,

[0048] Collect high-definition video data under construction scene, covering various construction actions (such as high-altitude operation, welding, carrying), the video should cover different time periods (day / night), different weather conditions (sunny / rainy) and different angles, in order to enhance data diversity. All videos should be standard resolution (such as 1080p) to ensure that the image quality meets the modeling requirements.

[0049] The video is processed by frame extraction using the video processing tool ffmpeg, and the frame extraction frequency is recommended to be set to 1-3 frames per second to avoid inter-frame information redundancy. The original video is disassembled into a series of independent image files, each image corresponding to a picture at a certain time in the video, retaining complete construction action information.

[0050] Each frame of image is labeled using the open source image labeling tool LabelImg, and the labeled objects are workers, safety helmets, reflective vests, aerial work equipment, welding flames, and carrying objects. The labeling format uses the standard format supported by the YOLO series model, and each labeling file records the target class number and normalized bounding box coordinates. All labeling results need to be reviewed by multiple people to ensure labeling accuracy and consistency.

[0051] The labeled video images are divided into a training set (70%) and a validation set (30%), and a random shuffle strategy is used to ensure balanced distribution of each image sample.

[0052] S2, improve and train YOLO using the labeled video data to obtain an improved YOLO model, specifically,

[0053] YOLO includes YOLOv3, YOLOv5, and YOLOv8 target detection models, and YOLOv3 is selected for improvement in this embodiment.

[0054] Download the YOLOv3 open source project code from the official GitHub repository and install the development dependencies such as Python 3.7, PyTorch 1.8, and OpenCV. Load the YOLOv3 pre-trained weight file yolov3.weights for subsequent transfer learning initialization to improve convergence speed.

[0055] Check the YOLOv3 prototype model structure, the backbone network is Darknet-53, and the output three-level feature maps are C3 layer: resolution is 1 / 8 of the input image, channel number is 256. C4 layer: resolution is 1 / 16 of the input image, channel number is 512. C5 layer: resolution is 1 / 32 of the input image, channel number is 1024.

[0056] Create a new Python unit file channel_attention.py, and define the ChannelAttention class in the Python unit file to implement the following function flow, that is, input the labeled video image into Darknet-53 to get the output feature maps of C3, C4 and C5 layers. Taking C3 as an example, the feature map shape is (B, 256, / 8, where B is batch size, 256 is the number of channels, H and W are the height and width of the feature map respectively, and the global average pooling is performed on the feature map to generate a vector with shape (B, C, 1, 1). Here, "global" means that the average value is taken for all spatial positions (i.e., all HxW pixels) of each channel. The formula for each channel is specifically,

[0057] ;

[0058] where, represents the feature map, represents the global average value of the c-th channel, represents the height of the feature map, represents the width of the feature map, represents the channel;

[0059] Two fully connected layers are used to compress and expand the (B, C, 1, 1) vector, and the compression ratio is set to 16. First, the channel dimension is reduced from C to C / 16 through a fully connected layer, and then the channel dimension is restored to C dimension through another fully connected layer, and the output result is a vector with shape (B, C), denoted as logits, which is the original value of the channel after processing by the fully connected layer. The Sigmoid function is applied to each element in this logits, specifically,

[0060] ;

[0061] where, represents the weight of the c-th channel, represents the weight, represents the Sigmoid function, represents the original value of the c-th channel after processing by the fully connected layer; The weight is multiplied by the C3 feature map channel by channel to complete the channel weighting (attention weight map). The purpose of this is to enhance important channel features and suppress unimportant features, thereby improving the network's attention to key information. In addition, the above channel attention method is applied to the C4 and C5 level feature maps in turn to improve the key channel expression ability of each level feature map. The channel attention mechanism enhances the expression ability of the channel, improves the recognition accuracy of small scale targets, the spatial attention mechanism strengthens the response of the target region, improves the posture recognition ability of construction personnel in complex background, and the BiFPN realizes bidirectional feature propagation, avoiding the information loss problem caused by single direction propagation in traditional FPN.

[0062]

[0063] ​​Create a new Python file bifpn.py to define the Bi-directional Feature Pyramid Network. In the bifpn.py file, define the class BiFPNBlock, which receives a set of multi-level feature maps as input (C3, C4, and C5). The up-sampling path starts from the high-level feature map (e.g., C5) to enlarge the spatial size of the high-level feature map C5 to the low-level feature map (e.g., C4) for fusion, enhancing the detection ability of small targets. Use the image interpolation algorithm to enlarge the high-level feature map C5 so that the height and width of the enlarged feature map C5 are consistent with the corresponding low-level feature map C4, without changing the channel number of the feature map C5 and preserving the original feature dimension. Perform weighted summation of the interpolated enlarged high-level feature map C5 and the corresponding low-level feature map C4. The fused new feature map will have both high-level semantic information and low-level spatial details. Add a BatchNorm layer and a ReLU nonlinear activation function to the fused feature map to enhance the nonlinear expression ability and convergence speed of the model.

[0064] The down-sampling path starts from the low-level feature map (e.g., C3), which has a higher spatial resolution but weaker semantic abstraction ability. The purpose is to reduce the spatial size of the low-level feature map C3 to be consistent with the high-level feature map (e.g., C4) through a pooling operation, where the pooling kernel size is set to 3x3 and the step size is 2. Each pooling operation reduces the size of the feature map C3 by half until it is consistent with the size of the high-level feature map C4. The weighted summation of the down-sampled low-level feature map C3 and the corresponding high-level feature map C4 enhances the contextual understanding ability of the high-level feature map and improves the recognition accuracy of large targets in complex backgrounds.

[0065] To further enhance the feature fusion effect, multiple BiFPNBlocks are usually stacked 2 times, for example, the first BiFPNBlock, input: C3, C4, C5; output: fused new feature maps P3_out, P4_out, P5_out; the second BiFPNBlock, input: the output of the previous level P3_out, P4_out, P5_out; output: stronger feature maps Q3_out, Q4_out, Q5_out after further fusion; finally output a set of high-quality fused feature maps for the YOLOv3 detection head to use for target prediction.

[0066] Open the YOLOv3 model configuration file and locate the original feature fusion structure FPN. Delete the configuration items of the original FPN structure and replace them with the newly defined BiFPN.

[0067] The channel attention unit and the spatial attention unit constructed in the foregoing are sequentially inserted into the input end of the BiFPN, that is, the input feature map of each BiFPN fusion node first passes through the channel attention to the spatial attention and then enters the BiFPN. The structure of the Head part of YOLOv3 is modified, the output format of the new feature map is adapted, the output format is still [class number + 5] x (H x W), the original inference interface compatibility is maintained, and finally the model is named YOLOv3-AS-BiFPN, wherein A represents channel and spatial attention, and BiFPN represents a bidirectional feature pyramid network. A complete YOLOv3-AS-BiFPN model structure configuration file is saved.

[0068] The training hyperparameters of the YOLOv3 model are set, that is, the initial learning rate is set to 0.01, the Cosine decay is used, the batch size is set to 16-32 (determined according to the memory capacity), the optimizer is selected as SGD, the momentum is set to 0.9, the weight initialization uses the HeNormal method, and the training round number is set to 100 epochs. The training script train.py is started, the YOLOv3-AS-BiFPN model combined with the training set is specified to be trained by using the above saved attention enhancement mechanism and multi-scale fusion mechanism, TensorBoard is used to monitor the training process, the loss curve and mAP change are observed, the model performance is evaluated on the validation set, and the accuracy, recall rate and F1 score are recorded. After the training is completed, the YOLOv3-AS-BiFPN model file, that is, the improved YOLOv3 model, is automatically saved.

[0069] Through the collection of the labeled construction personnel operation video data and the improvement and training of the YOLOv3 model, the optimization of the YOLOv3 basic model is realized. Specifically, the channel attention unit, the spatial attention unit and the BiFPN (bidirectional feature pyramid network) are added, which enhances the attention of the YOLOv3 model to the relevant information and the small target detection capability. This not only improves the robustness and accuracy of the YOLOv3 model in the complex construction site environment, but also enables the model to more accurately identify the behavior actions of the construction personnel. Through this optimization measure, the improved YOLOv3 model finally obtained can reduce the false detection rate and the missed detection rate, and provides a solid foundation for subsequent behavior classification and trajectory analysis.

[0070] S3, input the video data of the construction personnel operation action to be recognized into the improved YOLO model, and combine a multi-target tracking algorithm to obtain the behavior category and the behavior trajectory of the construction personnel, specifically,

[0071] Multiple high-definition cameras are deployed in the working areas of the construction site, such as high-altitude working areas, hoisting areas, and entrances and exits. The cameras are fixedly installed to ensure that the coverage range of the pictures is reasonable and has no dead angles. All cameras transmit real-time video streams to the local edge computing device through the RTSP protocol. The multiple video streams are synchronized in time stamp to ensure the consistency of the picture timing between different cameras. The video stream is de-bounced to eliminate the unstable phenomenon of the picture caused by wind blowing and mechanical vibration.

[0072] The OpenCV library is used to read and format convert each frame of image in the video stream, and the resolution is uniformly adjusted to 416x416 pixels (adapted to the input requirements of YOLOv3) for standardization processing to reduce the recognition error caused by light changes. After preprocessing, the image is cached to the memory queue for frame calling by the YOLOv3 model.

[0073] The image is taken out from the image cache queue frame by frame and input to the improved YOLOv3 model. The YOLOv3 model outputs the position information (bounding box coordinates) of all construction personnel in the current image frame, the identity category (such as "person"), the confidence score, and the specific behavior category (such as "not wearing a safety helmet", "illegal climbing", "illegal lingering", "drinking water", "resting"). The output result is processed by non-maximum suppression (NMS) to remove overlapping repeated detection boxes and bind with the personnel ID to form a structured data record.

[0074] The DeepSORT tracker is initialized, specifically, the Kalman filter component in the DeepSORT tracker is set to predict the target state, the cosine similarity matching threshold (such as 0.3) is set, and the IOU matching threshold (such as 0.5) is set. Among them, the cosine similarity matching threshold is used to measure the similarity of the ReID features between the bounding box and the DeepSORT tracker. The IOU matching threshold is used to measure the degree of spatial overlap between the bounding box and the predicted box of the Kalman filter component.

[0075] The bounding box coordinates output by the YOLOv3 model are converted to the center coordinates+width and height format required by the DeepSORT tracker, and the position of each construction personnel in the current image frame is predicted through the predicted box of the Kalman filter component.

[0076] The converted bounding box coordinates are concatenated with the predicted construction worker positions, i.e., if the distance between two construction workers is less than a Mahalanobis distance matching threshold (e.g., 9.488), it is considered to be the same construction worker. The Mahalanobis distance matching threshold is a fixed value set based on statistical theory (chi-square distribution). According to the position, coordinates and timestamp of the construction worker tracked by the DeepSORT tracker, a complete behavior trajectory data is formed, and behavior trajectory features (average moving speed, trajectory curvature and construction area stay time) are extracted from the trajectory data. Through the extraction of behavior trajectory features, not only the work path of the construction worker can be described in detail, but also potential safety hazards can be found, such as whether to stay in a dangerous area for a long time. The role of this step is to provide an effective method to evaluate the behavior risk of construction workers, which helps to develop more scientific and reasonable safety strategies.

[0077] S4, applying quantitative theory to quantitatively analyze the behavior categories and behavior trajectories of construction workers, specifically,

[0078] A data table named behavior_risk_mapping is created in the database, including behavior_name: behavior name, such as "no safety helmet", "illegal climbing", "illegal stay", "drinking water" and "rest"; risk_level: risk level description, such as no safety helmet is level one risk, illegal climbing is level two risk, illegal stay is level three risk, drinking water and rest have no risk level; risk_score: risk value, such as level one corresponds to 0.1~0.2, level two corresponds to 0.2~0.4, and so on;

[0079] According to the identity ID, behavior category and timestamp of the construction worker obtained above, the number of occurrences of each type of behavior is counted for each construction worker within a fixed time period (e.g., 5 minutes). For example, if a construction worker appears 10 times "without safety helmet" within 5 minutes, the frequency of this behavior is recorded. For each behavior category, the corresponding risk value is found from the mapping table, and the number of occurrences of each behavior is multiplied by the risk value to obtain a contribution value. The contribution values of all behaviors of the construction worker are added to obtain a score reflecting the overall behavior risk level. In order to unify the scoring standard, the risk score is normalized to 0 to 1.

[0080] A fixed length sliding window (for example, 5 seconds) is set, which is slid every 1 second to ensure data continuity and real-time performance. Each sliding window contains the trajectory data in the recent period, i.e., the behavior trajectory feature. A set of weights is set for the behavior trajectory feature in each sliding window, for example, the average speed weight is 0.2, the trajectory curvature weight is 0.3, and the construction area residence time weight is 0.3. Then, the behavior trajectory score of the construction personnel in the sliding window is calculated according to the weight of the set behavior trajectory feature, specifically,

[0081] M ;

[0082] wherein M represents the behavior trajectory score of the construction personnel in the sliding window, represents the average speed weight, represents the average speed, represents the trajectory curvature weight, represents the trajectory curvature, represents the construction area residence time weight, represents the construction area residence time;

[0083] According to the above sliding window formula, the behavior trajectory scores of the construction personnel in all sliding windows are calculated in turn, and the behavior trajectory scores of the construction personnel in all sliding windows are added to obtain the overall behavior trajectory score of the construction personnel.

[0084] By applying the quantification theory to the behavior category and behavior trajectory feature of the construction personnel, a comprehensive behavior risk assessment system is constructed, and the complex behavior mode of the construction site is scientifically analyzed and accurately evaluated. The originally difficult-to-quantify construction personnel behavior is converted into a specific risk score, which greatly improves the objectivity and accuracy of safety management decisions.

[0085] S5, and the comprehensive safety score of the construction personnel operation action is obtained by combining the safety evaluation model constructed based on the fuzzy analytic hierarchy process, specifically,

[0086] A three-layer structure is constructed, namely a target layer, a criterion layer and an index layer. The target layer is set as "construction personnel behavior safety level assessment", and this layer only contains one node, i.e. the safety level to be finally output. The criterion layer divides the factors affecting the behavior safety of construction personnel into behavior risk degree and behavior trajectory stability. The index layer is behavior risk score and behavior trajectory score, and a safety assessment model is obtained. In the constructed safety assessment model, the weight of the behavior risk score is set as 0.6, and the weight of the behavior trajectory score is set as 0.4. A time decay factor is set to reflect the influence of the time distance between the behavior occurrence and the current time, for example, 0.0183. According to the weights of the risk score and the behavior trajectory score and the time decay factor, the decayed behavior risk score and the decayed behavior trajectory score are calculated in the safety assessment model, specifically,

[0087] ;

[0088] ;

[0089] wherein, represents the decayed behavior risk score, represents an index, represents the behavior risk score, represents the weight of the behavior risk score, represents the decayed behavior trajectory score, represents a time decay factor, represents the weight of the behavior trajectory score;

[0090] The decayed behavior risk score and the decayed behavior trajectory score are added to obtain a final comprehensive safety score.

[0091] The comprehensive safety score is divided into five levels, which are named as A level, B level, C level, D level and E level respectively, and the A level is the safest and the E level is the most dangerous. According to the business requirements, the score range of each level is set, for example, A level: [0.00, 0.02], B level: (0.02, 0.05], C level: (0.05, 0.10], D level: (0.10, 0.20] and E level: > 0.20. The comprehensive safety score of each construction personnel is matched to the corresponding level, when the comprehensive score of a construction personnel is 0.0295, it belongs to the interval (0.02, 0.05], which is mapped to the B level (relatively safe), and if the score is 0.21, it belongs to the E level (extremely high risk).

[0092] S6, according to the comprehensive safety score and the behavior safety level of the construction personnel, an unsafe behavior detection report is generated, specifically,

[0093] A structured behavior event list is established, recording the construction personnel ID, behavior occurrence time, behavior category, risk level, comprehensive safety score, behavior trajectory feature score, and video segment number.

[0094] The report generation conditions are set, for example, when a construction personnel has a behavior of high risk or above in a day, the behavior event list of the construction personnel is marked as needing to generate an unsafe behavior detection report.

[0095] The daily unsafe behavior supervision template is prepared. Extract the list of construction personnel who need to generate an unsafe behavior detection report, and obtain the construction personnel name, position, team, and construction personnel ID, behavior occurrence time, behavior category, risk level, comprehensive safety score, behavior trajectory feature score, and video segment number in the behavior event list according to the construction personnel list.

[0096] Open the daily unsafe behavior supervision template, and find the field names (behavior event record, highest risk level, and video number) reserved in the unsafe behavior supervision template in turn, extract the corresponding data from the behavior event list according to the field names, and fill in, for example, the behavior event list inserts the behavior event record, the risk level fills in the highest risk level, and the video number fills in the corresponding video index. After filling in, the final detection report reflecting the unsafe behavior of the construction personnel is obtained. Through automatic generation of detailed unsafe behavior detection reports, the safety management process of the construction site is highly optimized and standardized. This not only simplifies the work burden of safety management personnel, but also ensures the objectivity and consistency of the evaluation results, avoiding the deviation caused by human factors. More importantly, this report mechanism can timely and accurately reflect the behavior risk and potential safety hazards of each construction personnel, so that the management can make more scientific and reasonable prevention measures and improvement schemes according to the specific data.

[0097] The embodiment also provides a computer device suitable for the construction personnel unsafe behavior detection method based on improved YOLO, which comprises a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the construction personnel unsafe behavior detection method based on improved YOLO proposed in the above embodiment.

[0098] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0099] The embodiment also provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the construction worker unsafe behavior detection method based on the improved YOLO. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0100] To sum up, the application builds a comprehensive and efficient construction personnel unsafe behavior detection mechanism from data collection, model improvement to behavior recognition and evaluation. Firstly, the accuracy of the YOLOv3 model in the complex construction site environment is improved by deep improvement of the YOLOv3 model. Secondly, combined with an advanced multi-target tracking algorithm, the behavior trajectory of the construction personnel is monitored and recorded in real time. Not only various types of behavior actions can be accurately identified, but also potential safety hazards can be effectively captured. The behavior risk is quantitatively evaluated by using the quantification theory and the fuzzy analytic hierarchy process. The complex on-site behavior is converted into a specific risk score. The scientificity and operability of safety management are improved. A detailed unsafe behavior detection report is generated. The overall accuracy and efficiency of the construction personnel unsafe behavior detection are greatly improved.

[0101] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application. They should be covered in the scope of the claims of the present application.

Claims

1. A method for detecting unsafe behaviors of construction workers based on improved YOLO, characterized by: include, Collecting labeled video data, wherein the labeled video data includes action videos of construction workers performing construction work; Use the labeled video data to improve and train YOLO to obtain the improved YOLO model; The video data of the construction workers' working actions to be identified is input into the improved YOLO model, and combined with the multi-target tracking algorithm, the construction workers' behavior categories and behavior trajectory characteristics are obtained; The construction workers' behavior categories and behavior trajectory characteristics were quantitatively analyzed using quantitative theory, and the safety assessment model based on fuzzy analytic hierarchy process was used to evaluate the safety of the construction workers to obtain a comprehensive safety score. Generate an unsafe behavior detection report based on the construction workers' comprehensive safety scores and behavioral safety levels.

2. The method for detecting unsafe behaviors of construction workers based on improved YOLO according to claim 1, characterized in that: The collecting of labeled video data refers to labeling the action video of the construction workers during construction work by using the labeling tool LabelImg, and decomposing the labeled video into independent images by using the tool ffmpeg.

3. The method for detecting unsafe behaviors of construction workers based on improved YOLO as claimed in claim 2, characterized in that: The improved YOLO model refers to using the YOLO architecture as the basic target detection model, adding the attention enhancement mechanism and the multi-scale fusion mechanism to the YOLO model, and training and verifying it through labeled video data to obtain the improved YOLO model.

4. The method for detecting unsafe behaviors of construction workers based on improved YOLO as claimed in claim 3, characterized in that: The attention enhancement mechanism and multi-scale fusion mechanism are specifically: Perform global average pooling on each channel of the YOLO model to generate a channel descriptor. Use the fully connected layer and channel descriptor of the YOLO model to calculate the importance weight of each channel. Use the importance weight to weight each channel of the image in the YOLO model to obtain an attention weight map. Perform maximum pooling and average pooling along the channel of the feature map in the YOLO model to obtain two two-dimensional spatial descriptors. Use the convolutional layer of the YOLO model to merge the two descriptors to generate a spatial attention map. A BiFPN unit is constructed in the feature pyramid structure of the YOLO model, and the BiFPN unit is used to interact and weightedly fuse information between feature maps of different levels in the YOLO model.

5. The method for detecting unsafe behaviors of construction workers based on improved YOLO according to claim 4, characterized in that: The video data of the construction workers' working actions to be identified is input into the improved YOLO model, and combined with the multi-target tracking algorithm, the behavior categories and behavior trajectories of the construction workers are obtained, specifically, Real-time collection of video streams of each construction worker's activities at the construction site to be tested; After unifying the video stream into the input requirements of the YOLO model, it is input into the improved YOLO model to obtain the behavior categories of the construction workers; The DeepSORT multi-target tracking algorithm is used to extract the average moving speed, trajectory curvature and duration of stay in the construction area of ​​the construction workers in the video stream to obtain the behavioral trajectory characteristics of the construction workers.

6. The method for detecting unsafe behaviors of construction workers based on improved YOLO according to claim 5, characterized in that: The quantitative analysis of the construction workers' behavior categories and behavior trajectory characteristics is carried out using quantitative theory, specifically, Establish a behavior risk level mapping table, map the construction workers' behavior categories to the risk level mapping table and assign corresponding risk values; Count the number of occurrences of construction workers' behavior categories and calculate them based on the risk values ​​of the behavior categories to obtain the construction workers' behavior risk scores; The construction workers' behavior trajectory characteristics are extracted using a sliding window to obtain statistical indicators, which are then calculated using a linear weighted sum to obtain the construction workers' behavior trajectory score.

7. The method for detecting unsafe behaviors of construction workers based on improved YOLO according to claim 6, characterized in that: The above-mentioned safety assessment model based on fuzzy analytic hierarchy process is used to evaluate the construction workers' operation, and the comprehensive safety score of the construction workers' operation is obtained, specifically, The fuzzy analytic hierarchy process is used to sequentially construct the target layer, criterion layer and indicator layer to obtain the safety assessment model; The construction workers' behavioral risk scores and behavioral trajectory scores are input into the safety assessment model for weight allocation and consistency verification to obtain the weights of behavioral risks and behavioral trajectories; Based on the construction workers' behavioral risk scores, behavioral trajectory scores, and corresponding weights, combined with the time decay factor, a fusion calculation is performed to obtain the construction workers' comprehensive safety score; A behavioral safety level interval is preset, and the comprehensive safety score of the construction workers is mapped to the preset behavioral safety level interval to obtain the behavioral safety level of the construction workers.

8. The method for detecting unsafe behaviors of construction workers based on improved YOLO according to claim 7, characterized in that: The unsafe behavior detection report is generated based on the comprehensive safety score and behavior safety level of the construction workers, specifically, Input the construction workers' comprehensive safety scores and behavioral safety levels into the report generation unit to obtain a list of construction workers' behavioral events and their occurrence times; The behavioral event list and occurrence time information are input into the structured template engine to generate a construction worker unsafe behavior detection report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for detecting unsafe behavior of construction workers based on improved YOLO are implemented as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting unsafe behaviors of construction workers based on improved YOLO are implemented.

Images