Safety early warning method and system for construction site based on object monitoring

By constructing an image data set with label information in the safety management of the construction site and introducing CSPDarkNet53 convolution and progressive attention mechanism, the YOLOv5 model is optimized, which solves the problem of insufficient lighting impact and feature extraction capabilities, and improves the target detection Accuracy and robustness ensure the accuracy of safety warning of construction sites.

CN118942034APending Publication Date: 2024-11-12BEIJING NORTH STAR DIGITAL REMOTE SENSING TECH CO LTD
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202410983615.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the safety management of construction sites, the accuracy of image recognition technology in complex environments is limited and is easily affected by factors such as light, resulting in a decrease in the accuracy of early warning results. The traditional YOLO model has weak image feature extraction capabilities. Impacts the accuracy of target detection.

Method used

By building an image dataset with label information, introducing CSPDarkNet53 convolution and progressive attention mechanisms to the YOLOv5 network, optimizing the loss function, and using Retinex-Net to enhance the dark light image, improving the YOLOv5 detection model to improve the target detection Accuracy.

Benefits of technology

Improve the accuracy and robustness of the model for object detection, enhance the detection ability of small-scale targets and occluded targets, reduce the complexity of the model, ensure the image quality under poor lighting conditions, and improve the warning results. accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942034A_ABST
    Figure CN118942034A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of construction site target detection, and particularly relates to a construction site safety early warning method and system based on object monitoring, and the method comprises the following steps: training an improved YOLOv5 detection model through employing an image data set with label information; acquiring a real-time construction site image of the target monitoring and early warning area; taking the real-time construction site image as the input of the trained improved YOLOv5 detection model, and outputting the detection result of the real-time construction site image of the target monitoring and early warning area; when the illegal behavior exists in the detection result, early warning is carried out; the system comprises a data processing module, a model construction module, a model training module, an image acquisition module, a target detection module and a safety early warning module. According to the invention, rapid detection of the target in the construction site can be realized, early warning can be timely sent to workers according to the detection result, and safe construction of the construction site is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of construction site target detection, and in particular relates to a safety early warning method and system for a construction site based on object monitoring. Background Art

[0002] In the safety management of construction sites, supervisors are often required to constantly monitor the construction sites through video recording or patrolling to prevent dangers caused by construction workers not being in place or unreasonable allocation of construction workers, which often requires considerable human resources.

[0003] In recent years, with the rapid development of artificial intelligence, some image processing-based construction site safety warning solutions have been provided for the safety management of construction sites. Specifically, real-time monitoring image data of the construction site is collected by surveillance cameras, and computer vision and deep learning technologies are used to realize automatic monitoring and early warning of safety hazards, which has made certain progress.

[0004] However, the existing technology still has the following limitations: First, the current image recognition technology has limited accuracy in complex environments and is easily affected by factors such as lighting, which leads to reduced accuracy of the warning results; Second, in the technology of using computer vision and deep learning technology to realize automatic monitoring and early warning of safety hazards, the YOLO model is often used to detect targets in images. However, the traditional YOLO model has weak feature extraction capabilities for images when processing computer vision tasks, which affects the accuracy of the YOLO model in target detection. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a safety warning method and system for construction sites based on object monitoring, aiming to solve the technical problem that the accuracy of object monitoring and warning of the traditional YOLO model is relatively low in the field of construction site safety management.

[0006] To achieve the above object, the present invention provides the following technical solutions.

[0007] In one embodiment of the present invention, the present invention provides a safety early warning method for a construction site based on object monitoring, the safety early warning method comprising the following steps:

[0008] Construct an image dataset with label information for the target monitoring and early warning area;

[0009] In the feature extraction stage of the YOLOv5 network, CSPDarkNet53 convolution is introduced, and the progressive attention mechanism is added to the CSPDarkNet53 convolution. Then, the optimizer is selected and the loss function is optimized to obtain the improved YOLOv5 detection model.

[0010] Using an image dataset with label information to train the improved YOLOv5 detection model;

[0011] Obtain real-time construction site images of the target monitoring and early warning area;

[0012] Using the real-time construction site image as the input of the trained improved YOLOv5 detection model, and outputting the detection result of the real-time construction site image of the target monitoring and early warning area;

[0013] When there are violations in the detection results, the violations and the real-time construction site images of the target monitoring and early warning area are pushed to the management background for early warning.

[0014] As a further solution of the present invention, the step of constructing an image data set with label information in the target monitoring and early warning area includes:

[0015] Obtain image datasets of construction sites through field photography;

[0016] Converting images of the image data set into grayscale images, analyzing the histogram of the grayscale images, identifying the proportion of pixels with low grayscale values ​​therein, and treating images whose proportion exceeds a threshold as dark-light images, and the remaining images constitute the original image set;

[0017] Based on Retinex-Net, the dark-light images in the image data are enhanced, and the enhanced dark-light images are used as the enhanced image set;

[0018] Use the LabelImg annotation tool to mark the images in the original dataset and the enhanced image set, and add corresponding label information to the targets in the images.

[0019] As a further solution of the present invention, the step of enhancing the dark light image in the image data based on Retinex-Net includes:

[0020] The decomposition network Decom-Net of Retinex-Net is used to decompose the dark-light image to obtain the reflection component and illumination component of each channel of the dark-light image. When decomposing the illumination component of the dark-light image, the RTV regularization term is used to establish the objective function to keep the edge smoothness of the illumination component.

[0021] The constraint relationship between the reflection component and the illumination component is used to optimize the loss function of Retinex-Net. The loss function uses reconstruction loss, reflection component consistency loss, and illumination component smoothness loss.

[0022] The enhanced network Enhance-Net of Retinex-Net is used to enhance the illumination component, and the BM3D algorithm is used to denoise the reflection component. The normal illumination image is reconstructed by combining the enhanced illumination component.

[0023] As a further solution of the present invention, in optimizing the loss function of Retinex-Net, the loss function is expressed as:

[0024] L total =λ1L recon +λ2L ir +λ3L is (1)

[0025]

[0026] L ir =||R ag -R zc ||1 (3)

[0027]

[0028] In the formula, λ ij Represents the reconstruction coefficient, R i is the reflection component; I j is the illumination component; S j Represented as a decomposed complete image; represents the gradient operator, λ g is the balance coefficient.

[0029] As a further solution of the present invention, the CSPDarkNet53 convolution is introduced in the YOLOv5 network feature extraction stage, including:

[0030] The YOLOv5 network feature extraction backbone network with CSPDarkNet53 convolution contains multiple C3 modules, each of which contains a convolutional layer, a residual component, and a loss calculation;

[0031] The input image is input into the upper and lower branches respectively: the upper branch includes a CBL module, multiple GhostBottlenecks with a step size of 1 and a GhostBottleneck group with a step size of 2; the lower branch sets a GhostBottleneck with a step size of 2; the output channels of the upper and lower branches are kept consistent, and the results of the two branches are added by concat to form the YOLOv5 network feature extraction backbone network.

[0032] As a further solution of the present invention, the progressive attention mechanism is added to the convolution of CSPDarkNet53, including:

[0033] Given the feature map θ of layer l-1 l-1 ∈R C×H×W , the spatial attention feature map α of the l-1 layer l-1 Calculate by formula (5) and formula (6): s

[0034] Ψ l-1 =w s *up(θ l-1 )2+b s (5)

[0035] α l-1 =Sigmoid(w m *Ψ l-1 +b m (6)

[0036] In the formula, * represents the convolution operation, w s 、w m Both represent convolution kernels, b s 、b m Both represent bias terms; up(·)2 represents an upsampling operation with a factor of 2;

[0037] Formula (5) performs convolution on the upsampled feature map with half the channel;

[0038] Formula (6) indicates that the feature map with halved channels is convolved to form a spatial position feature map, and the sigmoid function is used for normalization.

[0039] Feature map θ l-1 The spatial attention feature map α l-1 Guide θ l Selectively generate strong features:

[0040]

[0041] in, Spatial attention feature map α l-1 ∈[0, 1] 2H×2W ; The i-th feature map C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map.

[0042] As a further solution of the present invention, the YOLOv5 network feature extraction backbone network also includes an introduced sliding window branch, the sliding window branch includes 3 submodules, and the sliding window branch is one of the basic components for feature extraction in the YOLOv5 network, including a convolutional layer, a batch normalization layer, and an activation function, which is used to perform channel dimensionality reduction on the features extracted by the model;

[0043] Secondly, a global average pooling layer is added to perform spatial dimension reduction on the feature information extracted by the previous module;

[0044] Finally, add a normal convolution layer, which is equivalent to a fully connected layer, to extract the feature information after the pooling operation. After Sigmoid sorting, the final output result is obtained. The output represents the probability of the target existing in the sliding window.

[0045] As a further solution of the present invention, the loss function of the improved YOLOv5 detection model includes the confidence loss L obj , classification loss L clc , border loss L loc and sliding window loss L slid ;in:

[0046] L zong =ε1L obj +ε2L cls +ε3L loc +ε4L slid (8)

[0047]

[0048] In equations (8) and (9), ε1, ε2, ε3, and ε4 represent weight coefficients; M represents the number of images that are sequentially input into the model as windows; S i Indicates whether the i-th sliding window contains the true value of the region of interest, which is 1 if it is contained and 0 if it is not; Represents the output of the sliding window branch, which is the probability that the sliding window contains the detected target.

[0049] As a further solution of the present invention, the improved YOLOv5 detection model is trained using an image dataset with label information, including:

[0050] Collect image datasets with labeled information and divide the datasets into training sets and validation sets;

[0051] Performing preprocessing operations such as normalization and resizing on the images of the image dataset;

[0052] According to the characteristics of the target detection task, set the model parameters, including learning rate, batch size and number of iterations;

[0053] Use the training set to train the improved YOLOv5 detection model and update the model parameters through back propagation and gradient descent;

[0054] The model performance of the updated parameters is evaluated on the validation set, wherein the indicators for evaluating the model performance include accuracy, recall rate and F1 score. The model parameters are adjusted according to the evaluation results to complete the training of the improved YOLOv5 detection model.

[0055] In another embodiment of the present invention, the present invention provides a safety warning system for a construction site based on object monitoring, the safety warning system comprising:

[0056] A data processing module is used to construct an image dataset with label information in the target monitoring and early warning area;

[0057] The model building module is used to introduce CSPDarkNet53 convolution in the feature extraction stage of the YOLOv5 network, add the progressive attention mechanism after the CSPDarkNet53 convolution, select the optimizer, and optimize the loss function to obtain the improved YOLOv5 detection model;

[0058] A model training module, used to train the improved YOLOv5 detection model using an image dataset with label information;

[0059] An image acquisition module is used to acquire real-time construction site images of the target monitoring and early warning area;

[0060] A target detection module is used to use the real-time construction site image as the input of the trained improved YOLOv5 detection model, and output the detection result of the real-time construction site image of the target monitoring and early warning area;

[0061] The safety warning module is used to push the illegal behavior and the real-time construction site image of the target monitoring and warning area to the management background for early warning when there is illegal behavior in the detection results.

[0062] Compared with the prior art, the safety early warning method and system for construction sites based on object monitoring of the present invention have the following beneficial effects:

[0063] First, the present invention enhances the dark light images in the image data by introducing Retinex-Net, thereby avoiding the problem that the obtained images are often too dark overall or partially due to poor lighting conditions, ensuring the image quality of the model training data set, improving the performance of the target detection model, and improving the robustness of the model;

[0064] Second, in the construction of the detection model of the present invention, by introducing CSPDarkNet53 convolution in the feature extraction stage of the YOLOv5 network and adding the progressive attention mechanism to the CSPDarkNet53 convolution, the improved YOLOv5 detection model obtained has a strong feature extraction capability for images when used for target detection in construction sites, which greatly improves the accuracy of the model for target detection;

[0065] Third, the present invention adds the progressive attention mechanism to the convolution of CSPDarkNet53 to improve the detection accuracy and expressiveness of the detection model, making the detection ability of small-scale targets and occluded targets stronger. Among them, the progressive attention mechanism is useful for the safety monitoring of construction site personnel who are far away from the camera and whose bodies are occluded. Since the upper body area of ​​the construction personnel only occupies a small part of the collected image, the resolution of the features of the image will gradually decrease after repeated downsampling operations. The improved progressive attention mechanism module is particularly suitable for detecting the features of small-scale targets.

[0066] Fourth, the YOLOv5 network feature extraction backbone network of the present invention also introduces a sliding window branch, and adds a sliding window loss to the loss function of the improved YOLOv5 detection model, so that in the image sparse target scene, when the sliding window is a simple background sliding window where there is no target of interest, the complex deep network detection task is no longer performed, and the target can be quickly detected, which reduces the complexity of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0068] Figure 1 This is a flow chart of the implementation of the safety early warning method for construction sites based on object monitoring of the present invention;

[0069] Figure 2 It is a sub-flow chart of the safety early warning method of the construction site based on object monitoring of the present invention;

[0070] Figure 3 It is another sub-flow chart of the safety early warning method of the construction site based on object monitoring of the present invention;

[0071] Figure 4 It is another sub-flow chart of the safety early warning method for construction sites based on object monitoring of the present invention;

[0072] Figure 5 It is a structural block diagram of the safety early warning system of the construction site based on object monitoring of the present invention;

[0073] Figure 6 The present invention is a structural block diagram of a computer device. DETAILED DESCRIPTION

[0074] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0075] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0076] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0077] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0078] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0079] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0080] The technical solution of the embodiment of the present application is introduced below.

[0081] At present, in the safety management of construction sites, the accuracy of image recognition technology in complex environments is limited and is easily affected by factors such as lighting, resulting in reduced accuracy of early warning results. In the technology of using computer vision and deep learning technology to realize automatic monitoring and early warning of safety hazards, the YOLO model is often used to detect targets in images. However, the traditional YOLO model has weak feature extraction capabilities for images when processing computer vision tasks, which affects the accuracy of the YOLO model in target detection.

[0082] To solve the above problems, the present invention provides a safety warning method and system for a construction site based on object monitoring. The safety warning method of the present invention uses an image data set with label information to train the improved YOLOv5 detection model; obtains a real-time construction site image of a target monitoring and warning area; uses the real-time construction site image as an input of the trained improved YOLOv5 detection model, and outputs the detection result of the real-time construction site image of the target monitoring and warning area; and issues a warning when there is a violation in the detection result;

[0083] The safety warning system of the present invention comprises a data processing module, a model building module, a model training module, an image acquisition module, a target detection module and a safety warning module.

[0084] The present invention can realize rapid detection of targets in a construction site, and according to the detection results, timely issue early warnings to workers to ensure safe construction at the construction site.

[0085] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0086] In one embodiment of the present invention, by installing a photographing device in a designated area of ​​a construction site, optionally, the construction site is divided into a plurality of monitoring and early warning areas, each monitoring and early warning area is configured with one or more photographing devices, the photographing device is connected to a server via a network communication, and is used to upload images taken by the photographing device to the server in real time; the server is connected to a management background via a network communication, and the server is used to process and detect and identify images taken by the photographing device. When there are violations in the detection results of the image, the violations and the real-time construction site images of the target monitoring and early warning area are pushed to the management background for early warning.

[0087] The network may include various types of wired communication links or wireless communication links, for example: wired communication links include optical fibers, twisted pairs or coaxial cables, and wireless communication links include Bluetooth communication links and Wireless Fidelity (Wi-Fi) communication links.

[0088] Accordingly, the server may be hardware or software.

[0089] When the server is hardware, it can be various electronic devices with image processing capabilities and data processing capabilities, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the server is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here.

[0090] Specifically, Figure 1 As shown, the safety early warning method for construction sites based on object monitoring provided by the present invention comprises the following steps:

[0091] Step S101: constructing an image dataset with label information for a target monitoring and early warning area;

[0092] In step S101, the image data of the construction site is first collected by using a photographing device. The photographing device can be a fixed-position photographing device, or a photographing drone can be used to collect images from multiple angles and multiple time points at the construction site. In addition, the collected image data needs to be preprocessed, and the image preprocessing steps include but are not limited to image rotation, scaling, color conversion and other technologies to increase data diversity and improve the generalization ability of the model.

[0093] Step S102: Introduce CSPDarkNet53 convolution in the YOLOv5 network feature extraction stage, add the progressive attention mechanism to the CSPDarkNet53 convolution, select an optimizer, and optimize the loss function to obtain an improved YOLOv5 detection model;

[0094] Preferably, the present invention introduces CSPDarkNet53 convolution in the YOLOv5 network feature extraction stage, specifically including: the YOLOv5 network feature extraction backbone network introducing CSPDarkNet53 convolution includes multiple C3 modules, each C3 module includes a convolution layer, a residual component and a loss calculation;

[0095] The input image is input into the upper and lower branches respectively: the upper branch includes a CBL module, multiple GhostBottlenecks with a step size of 1 and a GhostBottleneck group with a step size of 2; the lower branch sets a GhostBottleneck with a step size of 2; the output channels of the upper and lower branches are kept consistent, and the results of the two branches are added by concat to form the YOLOv5 network feature extraction backbone network.

[0096] In the detection model construction of the present invention, by introducing CSPDarkNet53 convolution in the YOLOv5 network feature extraction stage and adding the progressive attention mechanism to the CSPDarkNet53 convolution, the obtained improved YOLOv5 detection model has a strong feature extraction capability for images when used for target detection in construction sites, which greatly improves the accuracy of the model for target detection.

[0097] Furthermore, in an embodiment of the present invention, the step of adding the progressive attention mechanism to the convolution of CSPDarkNet53 includes:

[0098] Given the feature map θ of layer l-1 l-1 ∈R C×H×W , the spatial attention feature map α of the l-1 layer l-1 Calculate by formula (5) and formula (6): s

[0099] Ψ l-1 =w s *up(θ l-1 )2+b s (5)

[0100] α l-1 =Sigmoid(w m *Ψ l-1 +b m ) (6)

[0101] In the formula, * represents the convolution operation, w s 、w m Both represent convolution kernels, b s , b m Both represent bias terms; up(·)2 represents an upsampling operation with a factor of 2;

[0102] Formula (5) performs convolution on the upsampled feature map with half the channel;

[0103] Formula (6) indicates that the feature map with halved channels is convolved to form a spatial position feature map, and the sigmoid function is used for normalization.

[0104] Feature map θ l-1The spatial attention feature map α l-1 Guide θ l Selectively generate strong features:

[0105]

[0106] in, Spatial attention feature map α l-1 ∈[0, 1] 2H×2W ; The i-th feature map C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map.

[0107] The present invention adds the progressive attention mechanism to the CSPDarkNet53 convolution to improve the detection accuracy and expressiveness of the detection model, making the detection ability of small-scale targets and occluded targets stronger. Among them, the progressive attention mechanism is useful for the safety monitoring of construction site personnel who are far away from the camera and whose bodies are occluded. Since the upper body area of ​​the construction personnel only occupies a small part of the collected image, the feature resolution of the image will gradually decrease after repeated downsampling operations. The improved progressive attention mechanism module is particularly suitable for detecting the features of small-scale targets.

[0108] Furthermore, the YOLOv5 network feature extraction backbone network also includes an introduced sliding window branch, which includes 3 submodules. The sliding window branch is one of the basic components for feature extraction in the YOLOv5 network, including a convolutional layer, a batch normalization layer, and an activation function, which is used to perform channel dimension reduction on the features extracted by the model;

[0109] Secondly, a global average pooling layer is added to perform spatial dimension reduction on the feature information extracted by the previous module;

[0110] Finally, add a normal convolution layer, which is equivalent to a fully connected layer, to extract the feature information after the pooling operation. After Sigmoid sorting, the final output result is obtained. The output represents the probability of the target existing in the sliding window.

[0111] Furthermore, the loss function of the improved YOLOv5 detection model includes the confidence loss L obj , classification loss L cls , border loss L loc and sliding window loss L slid ;in:

[0112] L zong =ε1L obj +ε2L clc +ε3L loc +ε4L slid (8)

[0113]

[0114] In equations (8) and (9), ε1, ε2, ε3, and ε4 represent weight coefficients; M represents the number of images that are sequentially input into the model as windows; S i Indicates whether the i-th sliding window contains the true value of the region of interest, which is 1 if it is contained and 0 if it is not; Represents the output of the sliding window branch, which is the probability that the sliding window contains the detected target.

[0115] Therefore, in an embodiment of the present invention, the YOLOv5 network feature extraction backbone network of the present invention also introduces a sliding window branch, and adds a sliding window loss to the loss function of the improved YOLOv5 detection model, so that in an image sparse target scene, when the sliding window is a simple background sliding window where there is no target of interest, complex deep network detection tasks are no longer performed, and rapid detection of the target can be achieved, reducing the complexity of the model.

[0116] Please continue reading Figure 1 , the safety warning method for construction sites based on object monitoring provided by the present invention further includes the following steps: Step S103: training the improved YOLOv5 detection model using an image data set with label information;

[0117] like Figure 4 As shown, in an embodiment of the present invention, the use of the image data set with label information to train the improved YOLOv5 detection model includes:

[0118] Step S401: collect an image data set with label information, and divide the data set into a training set and a validation set; optionally, for the division of the data set, the training set and the validation set are divided in a ratio of 8:2;

[0119] Step S402: performing preprocessing operations of normalizing and resizing the images in the image data set;

[0120] Among them, in step S402 of the embodiment of the present invention, the image file is read using the image processing library OpenCV, and the read image is resized according to the size required by the model input. For example, the image size is adjusted using the resize function of the image processing library. During the resizing process, the aspect ratio of the image is maintained to avoid image distortion. In the normalization process, the image pixel values ​​are converted into a distribution centered on their mean and standard deviation.

[0121] Further, in an embodiment of the present invention, the use of the image data set with label information to train the improved YOLOv5 detection model also includes:

[0122] Step S403: setting model parameters according to the characteristics of the target detection task, where the model parameters include learning rate, batch size and number of iterations;

[0123] Step S404: Use the training set to train the improved YOLOv5 detection model, and update the model parameters through back propagation and gradient descent;

[0124] Among them, in the embodiment of the present invention, in the back propagation updating model parameters, the image data is input, and the output result, that is, the predicted bounding box and category probability, is calculated through the forward propagation of the model; the difference between the predicted result and the true label is calculated to obtain the loss value. YOLOv5 usually uses a composite loss function, and the gradient of each parameter is calculated through the back propagation algorithm according to the loss value;

[0125] Furthermore, in the gradient descent update model parameters, the gradient calculated during the back propagation process represents the direction and magnitude of the parameter update.

[0126] Use the gradient descent algorithm to update the model parameters. The update formula is: Among them, θ represents the parameter, η represents the learning rate, Represents the gradient of the loss function J with respect to θ.

[0127] The present invention selects Adam as the optimizer. By further adjusting the parameters of the optimizer, the present invention uses a smaller learning rate for preheating at the beginning of training and gradually increases it to a predetermined value, which helps the model to converge stably.

[0128] Step S405: Evaluate the model performance of the updated parameters on the validation set, where the indicators for evaluating the model performance include accuracy, recall and F1 score. Adjust the model parameters according to the evaluation results to complete the training of the improved YOLOv5 detection model.

[0129] For further information, please refer to Figure 1 The safety early warning method for construction sites based on object monitoring provided by the present invention also includes the following steps:

[0130] Step S104: Acquire a real-time construction site image of the target monitoring and early warning area;

[0131] Step S105: using the real-time construction site image as the input of the trained improved YOLOv5 detection model, and outputting the detection result of the real-time construction site image of the target monitoring and early warning area;

[0132] Step S106: When there are violations in the detection results, the violations and the real-time construction site image of the target monitoring and early warning area are pushed to the management background for early warning.

[0133] Furthermore, in step S104 to step S106 of the present invention, images of the construction site at the required angles are obtained by means of a photographing device, and necessary preprocessing is performed on the images, such as resizing and normalization, to meet the model input requirements; the preprocessed images are input into a trained improved YOLOv5 model to perform target detection; the detection results are analyzed to identify whether there are any violations; early warning system: when a violation is detected, the information is pushed to the management background via an API or other communication methods.

[0134] like Figure 2 As shown, in an embodiment of the present invention, the steps of constructing an image data set with label information in a target monitoring and early warning area include:

[0135] Step S201: Acquire an image dataset of a construction site by field photography;

[0136] Step S202: converting the images in the image data set into grayscale images, analyzing the histogram of the grayscale images, identifying the proportion of pixels with low grayscale values ​​therein, and treating the images whose proportion exceeds a threshold as dark-light images, and the remaining images constitute the original image set;

[0137] Among them, a color image is usually composed of three color channels, RGB, while a grayscale image combines the information of these three channels into a single channel representing brightness. The conversion can be achieved by calculating the weighted sum of each pixel. The commonly used conversion formula is: Gray = 0.299R + 0.587G + 0.114B; for the rotation of the conversion tool, the image processing library OpenCV can be used for conversion; regarding the analysis of the histogram, the histogram is a chart representing the frequency of occurrence of each gray level in the image, which can be used to evaluate the contrast and brightness distribution of the image; the present invention constructs a histogram by counting the number of times each gray level appears in the image, and further uses the Matplotlib visualization library to draw the histogram;

[0138] Furthermore, the present invention determines the dark-light image by further identifying the ratio of low gray-value pixels, wherein low gray-value generally refers to darker pixels in the gray-scale image, whose gray-value is lower; the frequency of occurrence of gray levels below a certain threshold in the histogram is calculated, and this frequency is divided by the total number of pixels in the image to obtain the ratio; and a threshold is further set according to demand, and when the ratio of low gray-value pixels exceeds this threshold, the image is considered to be a dark-light image.

[0139] Furthermore, since the dark-light image also retains rich feature information, target detection in the dark-light image is also necessary. Therefore, in order to make full use of the dark-light image, the present invention further includes:

[0140] Step S203: performing enhancement processing on the dark-light images in the image data based on Retinex-Net, and taking the enhanced dark-light images as an enhanced image set;

[0141] Step S204: Use the LabelImg annotation tool to mark the objects in the original data set and the enhanced image set, and add corresponding label information to the objects in the images.

[0142] Since dark light images will affect the accuracy of the model's target detection, the selected dark light images are enhanced, such as Figure 3 As shown, in an embodiment of the present invention, the step of enhancing the dark light image in the image data based on Retinex-Net includes:

[0143] Step S301: Decom-Net, a decomposition network of Retinex-Net, is used to decompose the dark-light image to obtain the reflection component and illumination component of each channel of the dark-light image; when decomposing the illumination component of the dark-light image, the RTV regularization term is used to establish the objective function to keep the edge smoothness of the illumination component;

[0144] Step S302: Optimizing the loss function of Retinex-Net using the constraint relationship between the reflection component and the illumination component. The loss function uses reconstruction loss, reflection component consistency loss, and illumination component smoothing loss.

[0145] Step S303: Enhance the illumination component using the Enhance-Net of Retinex-Net, denoise the reflection component using the BM3D algorithm, and reconstruct the normal illumination image by combining the enhanced illumination component. In the loss function of optimizing Retinex-Net, the loss function is expressed as:

[0146] L toal =λ1L recon +λ2L ir +λ3L is (1)

[0147]

[0148] L ir =||R ag -R zc ||1 (3)

[0149]

[0150] In the formula, λ ij Represents the reconstruction coefficient, R i is the reflection component; I jis the illumination component; S j Represented as a decomposed complete image; represents the gradient operator, λ g is the balance coefficient.

[0151] The traditional YOLOv5 model does not consider the impact of dark light environment on target detection in construction sites. The dark light image obtained in the dark light environment will affect the accuracy of the YOLOv5 model in detecting targets in the image. Therefore, in order to reduce the impact of the dark light environment on the detection of targets using the model, the present invention introduces Retinex-Net to enhance the dark light images in the image data, thereby avoiding the problem that the obtained images are often overall or partially too dark due to poor lighting conditions, ensuring the image quality of the model training data set, improving the performance of the target detection model, and also improving the robustness of the model.

[0152] like Figure 5 As shown, in another embodiment of the present invention, the present invention provides a safety warning system for a construction site based on object monitoring, and the safety warning system 500 includes:

[0153] Data processing module 501, used to construct an image data set with label information in the target monitoring and early warning area;

[0154] The model building module 502 is used to introduce CSPDarkNet53 convolution in the YOLOv5 network feature extraction stage, and add the progressive attention mechanism after the CSPDarkNet53 convolution, select an optimizer, and optimize the loss function to obtain an improved YOLOv5 detection model;

[0155] A model training module 503 is used to train the improved YOLOv5 detection model using an image dataset with label information;

[0156] Image acquisition module 504, used to acquire real-time construction site images of the target monitoring and early warning area;

[0157] The target detection module 505 is used to use the real-time construction site image as the input of the trained improved YOLOv5 detection model, and output the detection result of the real-time construction site image of the target monitoring and early warning area;

[0158] The safety warning module 506 is used to push the illegal behavior and the real-time construction site image of the target monitoring and warning area to the management background for warning when there is illegal behavior in the detection results.

[0159] like Figure 6As shown, in a third aspect of the embodiment of the present invention, a computer device is further provided. The computer device 600 includes a memory 603 and a processor 601. The memory 603 stores a computer program 602. When the computer program 602 is executed by the processor 601, the safety early warning method for the construction site based on object monitoring of any one of the above embodiments is implemented;

[0160] Among them, the safety early warning method of the construction site based on object monitoring includes:

[0161] Step S101: constructing an image dataset with label information for a target monitoring and early warning area;

[0162] Step S102: Introduce CSPDarkNet53 convolution in the YOLOv5 network feature extraction stage, add the progressive attention mechanism to the CSPDarkNet53 convolution, select an optimizer, and optimize the loss function to obtain an improved YOLOv5 detection model;

[0163] Step S103: using the image data set with label information to train the improved YOLOv5 detection model;

[0164] Step S104: Acquire a real-time construction site image of the target monitoring and early warning area;

[0165] Step S105: using the real-time construction site image as the input of the trained improved YOLOv5 detection model, and outputting the detection result of the real-time construction site image of the target monitoring and early warning area;

[0166] Step S106: When there are violations in the detection results, the violations and the real-time construction site image of the target monitoring and early warning area are pushed to the management background for early warning.

[0167] As a non-volatile computer-readable storage medium, memory 603 can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the program instructions / modules corresponding to the safety warning method for construction sites based on object monitoring in the embodiment of the present application.

[0168] The memory 603 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created by the use of a safety warning method for a construction site based on object monitoring, etc.

[0169] In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the local module via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0170] The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of a computer device.

[0171] In this embodiment, the processor is used to run the program code stored in the memory or process data. The processors of the multiple computer devices of the computer device in this embodiment execute various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory, that is, to implement the steps of the safety early warning method for the construction site based on object monitoring of the above method embodiment.

[0172] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0173] Finally, it should be noted that the computer-readable storage medium (e.g., memory) herein may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. As an example and not by way of limitation, a non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. A volatile memory may include a random access memory (RAM), which may act as an external cache memory. As an example and not by way of limitation, RAM may be obtained in a variety of forms, such as synchronous RAM (DRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to include, but are not limited to, these and other suitable types of memory.

[0174] The various exemplary logical blocks, modules, and circuits described in conjunction with the disclosure herein may be implemented or executed using a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components that are designed to perform such functions. A general purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and / or any other such configuration.

[0175] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0176] It should be understood that, as used herein, the singular form "a" or "an" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the items listed in association. The serial numbers of the embodiments disclosed in the above embodiments of the present invention are for description only and do not represent the advantages and disadvantages of the embodiments.

[0177] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the embodiments of the present invention as above, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A safety early warning method for construction sites based on object monitoring, characterized in that: The safety early warning method comprises the following steps: Construct an image dataset with label information for the target monitoring and early warning area; In the feature extraction stage of the YOLOv5 network, CSPDarkNet53 convolution is introduced, and the progressive attention mechanism is added to the CSPDarkNet53 convolution. Then, the optimizer is selected and the loss function is optimized to obtain the improved YOLOv5 detection model. Using an image dataset with label information to train the improved YOLOv5 detection model; Obtain real-time construction site images of the target monitoring and early warning area; Using the real-time construction site image as the input of the trained improved YOLOv5 detection model, and outputting the detection result of the real-time construction site image of the target monitoring and early warning area; When there are violations in the detection results, the violations and the real-time construction site images of the target monitoring and early warning area are pushed to the management background for early warning.

2. The safety early warning method for construction sites based on object monitoring according to claim 1 is characterized in that: The steps to construct an image dataset with label information for the target monitoring and early warning area include: Obtain image datasets of construction sites through field photography; Converting images of the image data set into grayscale images, analyzing the histogram of the grayscale images, identifying the proportion of pixels with low grayscale values ​​therein, and treating images whose proportion exceeds a threshold as dark-light images, and the remaining images constitute the original image set; Based on Retinex-Net, the dark-light images in the image data are enhanced, and the enhanced dark-light images are used as the enhanced image set; Use the LabelImg annotation tool to mark the images in the original dataset and the enhanced image set, and add corresponding label information to the targets in the images.

3. The safety early warning method for construction sites based on object monitoring according to claim 2 is characterized in that: The steps of enhancing the dark light image in the image data based on Retinex-Net include: The decomposition network Decom-Net of Retinex-Net is used to decompose the dark-light image to obtain the reflection component and illumination component of each channel of the dark-light image. When decomposing the illumination component of the dark-light image, the RTV regularization term is used to establish the objective function to keep the edge smoothness of the illumination component. The constraint relationship between the reflection component and the illumination component is used to optimize the loss function of Retinex-Net. The loss function uses reconstruction loss, reflection component consistency loss, and illumination component smoothness loss. The enhanced network Enhance-Net of Retinex-Net is used to enhance the illumination component, and the BM3D algorithm is used to denoise the reflection component. The normal illumination image is reconstructed by combining the enhanced illumination component.

4. The safety early warning method for construction sites based on object monitoring according to claim 2 is characterized in that: In optimizing the loss function of Retinex-Net, the loss function is expressed as: L total =λ1L recon +λ2L ir +λ3L is (1) L ir =|R ag -R zc ||1 (3) In the formula, λ ij Represents the reconstruction coefficient, R i is the reflection component I j is the illumination component S j Represented as a decomposed complete image; represents the gradient operator, λ g is the balance coefficient.

5. The safety early warning method for construction sites based on object monitoring according to claim 2 is characterized in that: The CSPDarkNet53 convolution is introduced in the YOLOv5 network feature extraction stage, including: The YOLOv5 network feature extraction backbone network with CSPDarkNet53 convolution contains multiple C3 modules, each of which contains a convolutional layer, a residual component, and a loss calculation; The input image is input into the upper and lower branches respectively: the upper branch includes a CBL module, multiple GhostBottlenecks with a step size of 1 and a GhostBottleneck group with a step size of 2; the lower branch sets a GhostBottleneck with a step size of 2; the output channels of the upper and lower branches are kept consistent, and the results of the two branches are added by concat to form the YOLOv5 network feature extraction backbone network.

6. The safety early warning method for construction sites based on object monitoring according to claim 2 is characterized in that: The progressive attention mechanism is added to the CSPDarkNet53 convolution, including: Given the feature map θ of layer l-1 l-1 ∈R C×H×W , the spatial attention feature map α of the l-1 layer l-1 Calculate by formula (5) and formula (6): s P l-1 =w s *up(θ l-1 )2+b s (5) a l-1 =Sigmoid(w m *P l-1 +b m ) (6) In the formula, * represents the convolution operation, w s 、w m Both represent convolution kernels, b s , b m Both represent bias terms; up(·)2 represents an upsampling operation with a factor of 2; Formula (5) performs convolution on the upsampled feature map with half the channel; Formula (6) indicates that the feature map with halved channels is convolved to form a spatial position feature map, and the sigmoid function is used for normalization. Feature map θ l-1 The spatial attention feature map α l-1 Guide θ l Selectively generate strong features: in, Spatial attention feature map α l-1 ∈[0, 1] 2H×2W ; The i-th feature map C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map.

7. The safety early warning method for construction sites based on object monitoring according to claim 2 is characterized in that: The YOLOv5 network feature extraction backbone network also includes an introduced sliding window branch, which includes 3 submodules. The sliding window branch is one of the basic components for feature extraction in the YOLOv5 network, including a convolutional layer, a batch normalization layer, and an activation function, and is used to perform channel dimension reduction on the features extracted by the model; Secondly, a global average pooling layer is added to perform spatial dimension reduction on the feature information extracted by the previous module; Finally, add a normal convolution layer, which is equivalent to a fully connected layer, to extract the feature information after the pooling operation. After Sigmoid sorting, the final output result is obtained. The output represents the probability of the target existing in the sliding window.

8. The safety early warning method for construction sites based on object monitoring according to claim 2 is characterized in that: The loss function of the improved YOLOv5 detection model includes the confidence loss L obj , classification loss L clc , border loss L loc and sliding window loss L slid ; in: L zong =ε1L obj +ε2L clc +ε3L loc +ε4L slid (8) In equations (8) and (9), ε1, ε2, ε3, and ε4 represent weight coefficients; M represents the number of images that are sequentially input into the model as windows; S i Indicates whether the i-th sliding window contains the true value of the region of interest, which is 1 if it is contained and 0 if it is not; Represents the output of the sliding window branch, which is the probability that the sliding window contains the detected target.

9. The safety early warning method for construction sites based on object monitoring according to claim 2 is characterized in that: The improved YOLOv5 detection model is trained using an image dataset with label information, including: Collect image datasets with labeled information and divide the datasets into training sets and validation sets; Performing preprocessing operations such as normalization and resizing on the images of the image dataset; According to the characteristics of the target detection task, set the model parameters, including learning rate, batch size and number of iterations; Use the training set to train the improved YOLOv5 detection model and update the model parameters through back propagation and gradient descent; The model performance of the updated parameters is evaluated on the validation set, wherein the indicators for evaluating the model performance include accuracy, recall rate and F1 score. The model parameters are adjusted according to the evaluation results to complete the training of the improved YOLOv5 detection model.

10. A safety warning system for construction sites based on object monitoring, characterized in that: The safety warning system is used to implement the safety warning method for construction sites based on object monitoring as described in any one of claims 1 to 9, and the safety warning system includes: A data processing module is used to construct an image dataset with label information in the target monitoring and early warning area; The model building module is used to introduce CSPDarkNet53 convolution in the feature extraction stage of the YOLOv5 network, add the progressive attention mechanism after the CSPDarkNet53 convolution, select the optimizer, and optimize the loss function to obtain the improved YOLOv5 detection model; A model training module, used to train the improved YOLOv5 detection model using an image dataset with label information; An image acquisition module is used to acquire real-time construction site images of the target monitoring and early warning area; A target detection module is used to use the real-time construction site image as the input of the trained improved YOLOv5 detection model, and output the detection result of the real-time construction site image of the target monitoring and early warning area; The safety warning module is used to push the illegal behavior and the real-time construction site image of the target monitoring and warning area to the management background for early warning when there is illegal behavior in the detection results.

Citation Information

Cited By

  • Road occupation construction detection method, system, equipment, medium and program product

    CN120259982A

  • Intelligent early warning method for tunnel blasting construction safety risk

    CN120451888A

  • Hazardous chemical substance pipeline detection and early warning system

    CN121141806A

  • Building construction potential safety hazard real-time monitoring system based on AI visual identification

    CN121415323A

  • Construction safety monitoring system based on big data

    CN122821489A