Intelligent construction site hazard source management method and system

Abstract

The invention discloses an intelligent construction site hazard source management method and system, and relates to the field of construction management, and the method comprises the steps: collecting a construction site image; the collected image is transmitted to an edge computing node, the image is preprocessed, image features are extracted, and the image features comprise the color, texture and shape of a target; according to the image features, constructing a convolutional neural network risk control model based on an attention mechanism; performing hazard source identification on each area by using a risk control model, wherein a hazard source identification result comprises the category and the position of a hazard source; the danger source types include no safety belt wearing and no reflective clothing wearing; and transmitting the dangerous source identification result to an edge computing node, generating a dangerous source distribution diagram of the construction site according to the category and the position of the dangerous source, and performing construction site dangerous source management according to the dangerous source distribution diagram. In view of low construction site hazard source identification precision in the prior art, the hazard source identification precision is improved through multi-dimensional feature representation and multi-task joint optimization.

Description

Technical Field

[0001] The present application relates to the field of construction management, and in particular to a smart construction site hazard source management method and system. Background Art

[0002] With the rapid development of the construction industry, the safety management of construction sites has become increasingly prominent. There are various potential danger sources at the construction site, such as falling objects, fire and explosion, mechanical injuries, etc., which seriously threaten the life safety of construction workers. In order to reduce the probability of safety accidents and improve the safety management level of construction sites, it is urgent to introduce advanced technical means to realize automatic identification and early warning of danger sources.

[0003] Traditional construction site safety management mainly relies on manual inspection and supervision, which is inefficient and easily affected by subjective factors. In recent years, with the development of computer vision and deep learning technology, intelligent hazard source identification methods have received widespread attention. By deploying video surveillance systems at construction sites, collecting real-time image data, and using image recognition algorithms to automatically detect hazard sources, the efficiency and reliability of safety management can be greatly improved.

[0004] However, the construction site environment is complex and changeable, with factors such as lighting changes, occlusion interference, and scale differences, which poses a huge challenge to hazard source identification. Existing hazard source identification methods are mainly based on shallow features such as color and texture, which are difficult to effectively characterize the diverse characteristics of hazard sources, resulting in insufficient recognition accuracy. Summary of the invention

[0005] In response to the problem of low accuracy in identifying hazard sources at construction sites in the prior art, the present application provides a smart construction site hazard source management method and system, which improves the accuracy of hazard source identification in complex scenarios through multi-dimensional feature representation and multi-task joint optimization.

[0006] The purpose of this application is achieved through the following technical solutions.

[0007] One aspect of the present application provides a method for managing hazard sources at a smart construction site, including: collecting images of the construction site; transmitting the collected images to an edge computing node, preprocessing the images, and extracting image features, where the image features include the color, texture, and shape of the target; constructing a convolutional neural network risk control model based on the attention mechanism based on the image features; using the risk control model to identify hazard sources in each area, where the hazard source identification results include the category and location of the hazard source; the hazard source category includes not wearing a safety belt and not wearing a reflective vest; transmitting the hazard source identification results to the edge computing node, generating a hazard source distribution map of the construction site based on the category and location of the hazard source, and managing the hazard sources at the construction site based on the hazard source distribution map.

[0008] The collected image is transmitted to the edge computing node, and the image is preprocessed, including: using a semantic-based image segmentation algorithm to semantically annotate the pixels in the image and divide the image into a human body area and a background area; using a target detection algorithm based on a region proposal network to generate candidate areas and perform classification and regression to detect human targets in the image; using an HSV color space statistical algorithm to perform histogram statistics on the human body area to obtain a feature vector reflecting the color distribution of the human body target; using a texture feature extraction algorithm based on a gray-level co-occurrence matrix to extract the texture feature vector of the human body target by calculating the gray-level co-occurrence matrix of the human body area, which is used to identify whether the human body target is not wearing a reflective vest; using an edge direction histogram algorithm to perform gradient direction statistics on the edge pixels in the human body area to obtain a feature vector reflecting the contour shape of the human body target, which is used to identify whether the human body target is not wearing a seat belt; the extracted color feature vector, texture feature vector and shape feature vector are combined as image features.

[0009] Furthermore, the image is segmented into a human body area and a background area, including: using a pre-trained semantic segmentation network to extract features from the image, the semantic segmentation network adopts an encoder-decoder structure, the encoder extracts image features through convolution and downsampling, and the decoder restores the resolution of the feature map through upsampling and jump connections; using a fully convolutional network to map the extracted features into a pixel-level category probability map; based on the pixel-level category probability map, a threshold segmentation method is used to segment the image into a human body area and a background area, the human body area corresponds to the area of ​​human body pixels in the category probability map, and the background area corresponds to the area of ​​non-human pixels in the category probability map.

[0010] Furthermore, the texture feature vector of the human target is extracted, including: converting the segmented human body area into a grayscale image, and quantizing the grayscale image according to a preset grayscale level L to obtain a quantized grayscale image; in the quantized grayscale image, selecting a target pixel point, setting a rectangular local window with the target pixel point as the center, and extracting the pixel grayscale value of the local window; calculating the grayscale co-occurrence matrix P(i,j,d,θ) in the local window, wherein P(i,j,d,θ) represents the probability that the grayscale levels of two pixels with a distance of d in the direction θ are i and j respectively; the direction θ includes four directions: horizontal, vertical, 45 degrees and 135 degrees; calculating the texture features according to the grayscale co-occurrence matrix to obtain a feature vector reflecting the local texture attributes of the human target, and the texture features include contrast C, energy ASM and entropy ENT; the contrast C is calculated by the following formula: Wherein, k is the contrast coefficient, and its value range is 1.5 to 2.5. Specifically, an adjustable contrast coefficient k is introduced. When the value of k is between 1.5 and 2.5, the contribution of pixels with large grayscale differences to the contrast can be increased. This helps to highlight the contrast difference between the reflective clothing and the non-reflective clothing in the human body area, and improves the accuracy of reflective clothing recognition.

[0011] Energy ASM is calculated by the following formula: Among them, r is the energy coefficient, and its value range is 1 to 2. Specifically, an adjustable energy coefficient r is introduced. When r is between 1 and 2, the weights of the elements near the diagonal of the gray-level co-occurrence matrix can be nonlinearly enhanced. This helps to quantify the consistency and repeatability of the texture in the human body area and improve the distinguishability of the texture features of the reflective clothing.

[0012] Entropy ENT is calculated by the following formula: Among them, L is the gray level, ranging from 16 to 256; P(i,j) is the element of the gray level co-occurrence matrix, which represents the probability that the gray values ​​of two pixels with a preset step length d distance in the preset direction θ are i and j respectively; i and j are the row and column numbers of the gray level co-occurrence matrix, ranging from 0 to L-1, representing the quantized pixel gray value. Specifically, the normalization factor 1 / log(L) is used instead of the common 1 / log2, where L is the gray level. Under this normalization method, the entropy value range is more stable and is less affected by the quantization level L. When L is between 16 and 256, the effectiveness of the entropy value can be maintained while reducing the amount of calculation.

[0013] Furthermore, a feature vector reflecting the contour shape of a human target is obtained, including: performing Gaussian filtering on the segmented human region image; performing edge detection on the filtered human region image using the Canny algorithm to obtain an edge image of the human target; performing morphological closing operation on the edge image; performing gradient calculation on the edge image after the closing operation using the Sobel operator to obtain the gradient amplitude Mag(x,y) and gradient direction Ang(x,y) of each edge pixel; dividing the gradient direction Ang(x,y) into M intervals, and counting the sum of the gradient amplitudes in each interval to obtain an M-dimensional gradient direction histogram, where the value of M is greater than or equal to 4; constructing a seat belt shape template based on seat belt detection prior information, and the seat belt shape template uses two parallel diagonals; calculating the similarity Sim(x,y) between the gradient direction histogram and the seat belt shape template to obtain a seat belt detection similarity map; judging whether the human target is not wearing a seat belt based on the seat belt detection similarity map.

[0014] Furthermore, the similarity Sim(x,y) between the gradient direction histogram and the seat belt shape template is calculated by the following formula: Where Hist(x,y) is the gradient direction,

[0015]

[0016] The value of the histogram at the (x, y) position represents the accumulated amplitude of the edge pixel points in the local area centered at (x, y) in the corresponding gradient direction interval; Temp(xu, yv) is the value of the seat belt shape template at the (xu, yv) position, which represents the value of the corresponding position after the template is offset by (u, v) units relative to (x, y), reflecting the gradient direction distribution of the edge of the seat belt; Hist mean and Temp mean are the means of Hist and Temp respectively; ε is a small positive constant used to avoid the denominator being zero, and its value range is 0.001 to 0.01; u and v are the offsets of the template in the x and y directions respectively, and their value range is 0 to the image width / height minus the template width / height.

[0017] Specifically, before calculating the correlation coefficient, the mean values ​​Histmean and Tempmean are respectively subtracted from the gradient direction histogram Hist and the shape template Temp to achieve feature centering. This normalization method can eliminate the brightness and contrast differences between the histogram and the template in different images and improve the matching stability. When calculating the correlation coefficient, a small positive constant ε (such as 0.001) is added to the denominator to avoid the situation where the denominator is zero when the histogram or template is a constant (variance is zero). The value of ε should be small enough to avoid significantly changing the magnitude relationship of the correlation coefficient, and is usually between 0.001 and 0.01. Adding a small positive constant avoids singular values ​​in the correlation coefficient calculation and improves the stability of the algorithm. The template offsets u and v in the horizontal and vertical directions are defined as variable parameters, and their value ranges are related to the size of the image and template. This variable search range can adapt to seat belt areas of different sizes and positions, improving the accuracy and robustness of matching. By using a variable template offset parameter, seat belts can be searched in a larger image area to adapt to changes in wearing position and size. Even if the seat belt is not worn properly or is blocked, reliable positioning can be achieved through local matching.

[0018] Furthermore, a convolutional neural network risk control model based on the attention mechanism is constructed, including: inputting image features into the convolutional neural network, and obtaining the semantic features of the image through convolution and pooling operations; setting an attention layer at the last layer of the convolutional neural network, and weightedly fusing the semantic features through the attention mechanism to obtain the semantic feature vector after attention enhancement; inputting the semantic feature vector after attention enhancement into the fully connected layer, and mapping the semantic feature vector to the category probability distribution of the hazard source through the softmax activation function; judging whether there is no seat belt or reflective clothing in the image according to the category probability distribution of the hazard source, and if so, outputting the category and location coordinates of the hazard source; constructing a loss function, optimizing the parameters of the convolutional neural network through back propagation, minimizing the loss function, and obtaining the trained risk control model.

[0019] Furthermore, the loss function expression is as follows: Loss = α × Loss clswl +β×Loss clsab +γ×Loss reg , where Loss is the total loss function, Loss clswl Loss for the category identification of not wearing reflective clothing, clsab Identify the loss for the category of not wearing a seatbelt, Loss reg is the regression loss of the hazard source location; α, β and γ are balance factors, which are used to adjust the contribution weights of the three losses to the total loss, and their values ​​range from 0 to 1;

[0020] Among them, N is the number of training samples, y 0i Indicates the true label of whether sample i wears reflective clothing, with a value of 0 or 1, y 1i Indicates the true label of whether sample i is not wearing reflective clothing, with a value of 0 or 1, and y 0i +y 1i =1; p 0i The probability that the risk control model predicts that sample i wears reflective clothing, p 1i is the probability that the risk control model predicts that sample i does not wear reflective clothing, and p 0i +p 1i =1; w0 and w1 are the balance weights of the two types of samples; Among them, y 2i Indicates the true label of whether sample i wears a seat belt, y 3i The true label indicating whether sample i is not wearing a seat belt; p 2i is the probability value of sample i wearing a seat belt predicted by the risk control model, p 3i is the probability value predicted by the risk control model that sample i does not wear a seat belt; w2 and w3 are the weight coefficients of the two types of samples; Among them, N is the number of training samples, IOUi is the prediction box t i 'With the real frame i The intersection-and-union ratio, Dist(t i -t i ') is the Euclidean distance between the center of the predicted box and the center of the real box; c is the diagonal distance of the minimum closure area covering both the predicted box and the real box;

[0021] It represents the consistency measure between the aspect ratio of the predicted box and the aspect ratio of the real box; α is a trade-off factor used to balance the contribution of the IOU term and the aspect ratio consistency term to the loss, and its value range is 0 to 1.

[0022] Furthermore, the hazard sources at the construction site are managed according to the hazard source distribution map, including: transmitting the hazard source categories and location coordinates identified by the risk control model to the edge computing node; in the edge computing node, grouping the hazard sources according to their categories, dividing the hazard sources without reflective clothing into the first group, and dividing the hazard sources without wearing safety belts into the second group; according to the first group of hazard sources, marking red signs at the corresponding positions in the construction site plan, and the red signs are in the form of triangles, indicating hazard sources without wearing reflective clothing; according to the second group of hazard sources, marking yellow signs at the corresponding positions in the construction site plan, and the yellow signs are in the form of circles, indicating hazard sources without wearing safety belts; and the construction site plan marked with hazard source signs is used as a hazard source distribution map.

[0023] Another aspect of the present application also provides a smart construction site hazard source management system for executing a smart construction site hazard source management method of the present application.

[0024] Compared with the prior art, the advantages of this application are:

[0025] The grayscale co-occurrence matrix quantitatively describes the second-order statistical characteristics of the image texture, such as roughness, regularity, and direction, by counting the probability of pixel co-occurrence in different directions and step sizes. Quantifying the texture features into three indicators, contrast, energy, and entropy, to form a texture feature vector, can fully reflect the texture differences of human clothing and provide a criterion for identifying whether reflective clothing is worn. Since reflective clothing has a unique grid-like texture structure that forms a regular texture pattern, while the texture of ordinary clothing is relatively random and messy, the use of the grayscale co-occurrence matrix to characterize the texture features can clearly distinguish whether reflective clothing is worn, thereby improving the recognition accuracy of not wearing reflective clothing;

[0026] Performing gradient analysis on the edge of the human body contour, statistically analyzing the distribution of gradient amplitudes in different directions, and constructing an edge direction histogram can characterize the shape characteristics of the human body contour. By designing a seat belt shape template with "two parallel diagonals" and matching the histogram with the template, it is possible to determine whether the person is wearing a seat belt. When a seat belt is worn, the diagonal shape feature of the seat belt is superimposed on the human body contour, while this shape feature is missing when not worn. Template matching can highlight the shape difference between the two situations, thereby improving the recognition accuracy of not wearing a seat belt;

[0027] The multi-task learning paradigm of softmax classification and position regression is used. The category and location information of the hazard source are considered at the same time. "Not wearing a seat belt" and "Not wearing a reflective vest" are used as the recognition targets. The bounding box coordinates are used as the position representation. By jointly optimizing the classification loss and position loss, a coupling association between category discrimination and spatial positioning is established. It requires both the prediction of the existence of the hazard source and the accurate bounding box position, avoiding the information loss caused by a single task and improving the overall recognition and positioning accuracy of the hazard source.

[0028] The design combines the joint optimization goal of category loss and position loss, introduces sample balance and prediction box aspect ratio constraints, and proposes an improved CIOU loss, which enhances the convergence and generalization performance of model training. Sample balance alleviates the imbalance problem of positive and negative sample ratios in training data by setting high weights for positive samples such as "not wearing reflective clothing" and "not wearing seat belts"; the prediction box aspect ratio constraint penalizes the difference in aspect ratio between the prediction box and the real box, making the shape of the prediction box more consistent with the target object; CIOU loss introduces the center distance penalty term and aspect ratio consistency constraint term on the basis of the original IOU loss, which accelerates the overlap process of the prediction box and the real box, and effectively improves the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present application will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0030] Figure 1 It is a schematic diagram of an application scenario of a smart construction site hazard source management method shown in the present application;

[0031] Figure 2 is an exemplary flow chart of a smart construction site hazard source management method according to the present application;

[0032] Figure 3 It is a schematic diagram of the detection effect shown in this application. DETAILED DESCRIPTION

[0033] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0034] like Figure 1 and Figure 2 As shown, images of the construction site are collected; the collected images are transmitted to the edge computing node, the images are preprocessed, and image features are extracted. The image features include the color, texture, and shape of the target; according to the image features, a convolutional neural network risk control model based on the attention mechanism is constructed; the risk control model is used to identify the hazard sources in each area, and the hazard source identification results include the category and location of the hazard source; the hazard source categories include not wearing a safety belt and not wearing a reflective vest; the hazard source identification results are transmitted to the edge computing node, and a hazard source distribution map of the construction site is generated according to the category and location of the hazard source, and the hazard source management of the construction site is carried out according to the hazard source distribution map.

[0035] In this embodiment, in order to obtain high-quality image data, the system uses advanced panoramic camera equipment. Specifically, an integrated ball camera with a high-definition resolution of 16 million pixels is used, and a single camera has a resolution of 4 million pixels, which can clearly capture the detailed information of the construction site. The camera equipment supports 360-degree panoramic imaging, which can cover all areas of the construction site without dead angles and leave no monitoring blind spots. At the same time, the equipment is equipped with a 100-meter infrared fill light function, which can recognize images even in low light conditions, ensuring the continuity and reliability of data collection. Through continuous recording and snapshot, the system can fully control the progress information of the construction site. It is worth noting that a single set of panoramic equipment is installed on a tower crane at a height of 20 meters, which can cover a wide area of ​​70,000 square meters. This deployment method maximizes the scope of data collection and reduces the number of equipment.

[0036] The collected image is transmitted to the edge computing node, and the image is preprocessed to extract image features, which include the color, texture and shape of the target. The semantic-based image segmentation algorithm is used to semantically annotate the pixels in the image and segment the image into the human body area and the background area. Specifically, the size of the input image I is W×H×3, where W=1920 and H=1080, indicating that the width and height of the image are 1920 pixels and 1080 pixels respectively. 3 indicates that the image has three color channels, RGB. Image I is input into the pre-trained semantic segmentation network for feature extraction. The network adopts an encoder-decoder structure: Encoder part: The encoder consists of N=5 convolution blocks. The first convolution block contains two 3x3 convolution layers with an output channel number of 64; and a 2x2 maximum pooling layer with a step size of 2. After the input image passes through the first convolution block, the size of the feature map becomes 960×540×64. The second convolution block contains two 3x3 convolution layers with an output channel number of 128; and a 2x2 maximum pooling layer with a step size of 2. The size of the feature map becomes 480×270×128. The third convolution block contains two 3x3 convolution layers with an output channel number of 256; and a 2x2 maximum pooling layer with a stride of 2. The size of the feature map becomes 240×135×256. The fourth convolution block contains two 3x3 convolution layers with an output channel number of 512; and a 2x2 maximum pooling layer with a stride of 2. The size of the feature map becomes 120×68×512. The fifth convolution block contains two 3x3 convolution layers with an output channel number of 512; and a 2x2 maximum pooling layer with a stride of 2. The size of the feature map becomes 60×34×512. The output of the encoder is a set of feature maps of size 60×34×512.

[0037] Decoder part: The decoder consists of N=5 upsampling blocks. The first upsampling block contains a 2x2 upsampling layer, which increases the resolution of the feature map from 60×34 to 120×68; and two 3x3 convolutional layers with an output channel number of 512. The upsampled feature map is concatenated with the output of the fourth convolutional block of the encoder, and the size of the feature map becomes 120×68×1024. The second upsampling block contains a 2x2 upsampling layer, which increases the resolution of the feature map from 120×68 to 240×135; and two 3x3 convolutional layers with an output channel number of 256. The upsampled feature map is concatenated with the output of the third convolutional block of the encoder, and the size of the feature map becomes 240×135×512. The third upsampling block contains a 2x2 upsampling layer, which increases the resolution of the feature map from 240×135 to 480×270; and two 3x3 convolutional layers with an output channel number of 128. The upsampled feature map is concatenated with the output of the second convolutional block of the encoder, and the size of the feature map becomes 480×270×256. The fourth upsampling block contains a 2x2 upsampling layer, which increases the resolution of the feature map from 480×270 to 960×540; and two 3x3 convolutional layers with an output channel number of 64. The fifth upsampling block contains a 2x2 upsampling layer, which increases the resolution of the feature map from 960×540 to 1920×1080; and two 3x3 convolutional layers with an output channel number C of 2, representing the two categories of human and background. The output of the decoder is a set of feature maps with a size of 1920×1080×2.

[0038] The size of the feature map F output by the decoder is 1920×1080×2, which means the width is 1920 pixels, the height is 1080 pixels, and the number of channels is 2. Each pixel in the feature map F corresponds to the feature values ​​of two categories (human body and background). The feature map F is input into the fully convolutional network for category probability mapping: the fully convolutional network consists of a 1x1 convolution layer with a convolution kernel size of 1×1, 2 input channels, and 2 output channels. The convolution layer performs a pixel-by-pixel convolution operation on the feature map F. For each pixel (x, y) on the feature map, the convolution kernel is element-wise multiplied with the feature values ​​of the two channels corresponding to it and the sum is calculated. The output of the convolutional layer is a 1920×

[0039] 1080×2 tensor S, each pixel point (x, y) in S contains two raw score values ​​S[x, y, 0] and S[x, y, 1], which respectively represent the probability of the pixel point (x, y) belonging to the human body and background categories. Perform Softmax normalization operation on the two raw score values ​​S[x, y, 0] and S[x, y, 1] of each pixel point (x, y) in tensor S to convert the raw scores into probability values:

[0040] Among them, P[x, y, 0] represents the probability that the pixel point (x, y) belongs to the human category, P[x, y, 1] represents the probability that the pixel point (x, y) belongs to the background category, and P[x, y, 0] + P[x, y, 1] = 1. After Softmax normalization, the category probability map P is obtained, and the size is P∈R 1920×1080×2 Each pixel point (x, y) in P corresponds to the probability value of two categories (human and background), P[x, y, 0] represents the probability of the human category, and P[x, y, 1] represents the probability of the background category.

[0041] Perform threshold segmentation on the category probability map P to obtain the segmentation results of the human body area and the background area: Set the threshold T to 0.6, that is, if the probability of a pixel belonging to the human body category is greater than 0.6, it is marked as a human body pixel; otherwise, it is marked as a background pixel. Traverse each pixel point (x, y) in the category probability map P: Get the probability value P[x, y, 0] that the pixel point (x, y) belongs to the human body category. If P[x, y, 0]>T, mark the pixel point (x, y) as a human body pixel and set M[x, y]=1; otherwise, mark the pixel point (x, y) as a background pixel and set M[x, y]=0. Repeat the above steps until all pixels in P are traversed. After the traversal is completed, a binary segmentation mask M of size 1920×1080 is obtained: M is a two-dimensional matrix with the same size as the width and height of the input image I. Each element M[x, y] in M ​​represents the category label of the pixel point (x, y) at the corresponding position in image I. If M[x, y] = 1, it means that the pixel (x, y) belongs to the human body area; if M[x, y] = 0, it means that the pixel (x, y) belongs to the background area. Output segmentation results, including: category probability map P, size 1920×1080×2, P[x, y, 0] represents the probability that the pixel (x, y) belongs to the human body category, P[x, y, 1] represents the probability that the pixel (x, y) belongs to the background category. P can be used to visualize the segmentation results, such as rendering the human body area in red and the background area in black. Segmentation mask M, size 1920×1080, M[x, y] = 1 represents the pixel (x, y) belongs to the human body area, M[x, y] = 0 represents the pixel (x, y) belongs to the background area. M can be used for subsequent human target analysis and recognition tasks, such as extracting the bounding box of the human body area, calculating the size and position of the human body area, etc.

[0042] The target detection algorithm based on the Region Proposal Network (RPN) is used to generate candidate regions and perform classification and regression to achieve accurate detection of human targets in the image. Specifically, let the input image be I, and its size is W×H. The goal of the target detection algorithm is to find all human targets in image I and represent their positions and sizes with bounding boxes. Each bounding box can be described by a four-tuple (x, y, w, h), where (x, y) represents the center coordinates of the bounding box, and w and h represent the width and height of the bounding box, respectively. The RPN network receives the image I as input and extracts a multi-scale feature map F through a convolutional neural network (CNN). On the feature map F, the RPN network generates k candidate regions (anchor boxes) of different sizes and aspect ratios at each position by sliding windows. These candidate regions can be represented as (x i ,y i ,wi ,h i ), where i = 1, 2, ..., N, where N is the total number of candidate regions. For each candidate region, the RPN network predicts two types of information: the probability p of whether the region contains a human target i ; The position offset of this area relative to the real target bounding box (Δx i ,Δy i ,Δw i ,Δh i ). By setting a threshold (such as 0.7), the probability p i The higher candidate regions are retained to obtain a set of potential human target regions. Next, these potential target regions are input into the classification and regression modules for further processing. The classification module performs binary classification on each region to determine whether it actually contains a human target. The regression module fine-tunes the position of each region to generate more accurate target bounding box coordinates. In the training phase, the object detection model uses paired input images I and annotated bounding boxes B as samples, and optimizes the parameters of the RPN network and classification regression module by minimizing the classification loss and regression loss. The classification loss uses the binary cross entropy function The regression loss uses the SmoothL1 function In the inference stage, the input image I is passed through the trained object detection model to generate a set of predicted human object bounding boxes. In order to remove redundant and overlapping detection results, the non-maximum suppression (NMS) algorithm is usually used to retain the target boxes with the highest confidence and non-overlapping as the final output.

[0043] The HSV color space statistical algorithm is used to perform histogram statistics on the human body area to obtain the feature vector reflecting the color distribution of the human body target. Specifically, the human body area obtained by the target detection algorithm is R, and its size is W. r ×H r In order to extract the color features of the human target, the region R is first converted from the RGB color space to the HSV color space. The HSV color space consists of three channels: hue, saturation, and value, which is more in line with the perception characteristics of the human eye. The conversion formula is as follows:

[0044]

[0045] Among them, R, G, B represent the red, green, and blue channel values ​​of the pixel respectively, H, S, and V represent the hue, saturation, and brightness values ​​of the pixel respectively, HSV(x,y)=(H(x,y),S(x,y),V(x,y)), (x, y) represents the pixel coordinates in the area.

[0046] In the HSV space, a histogram is calculated for each channel of the human body region R. Assume that the value range of the hue channel H is [0, 180), and it is divided into N h The saturation channel S and the lightness channel V have a value range of [0, 255], which are divided into N intervals. s and N v Intervals. Count the number of pixels in each interval and get the histogram vectors of the three channels:

[0047] Among them, h i 、s i 、v i Respectively represent the number of pixels in the i-th interval. In order to eliminate the influence of scale difference and illumination change, the histogram vector is normalized to obtain the normalized feature vector: Finally, the feature vectors of the three channels are concatenated to form a vector with a length of (N h +N s +N v )’s color feature vector C feat :

[0048] Through the HSV color space statistical algorithm, the system can effectively extract the color distribution characteristics of human targets and form a compact and robust feature representation. This feature representation can capture the main color attributes of human targets, such as the bright colors of reflective clothing, and provide important discriminant information for subsequent hazard source identification. At the same time, histogram statistics and normalization processing also make the algorithm invariant to scale changes and illumination changes, improving the reliability and adaptability of feature extraction.

[0049] The texture feature extraction algorithm based on gray-level co-occurrence matrix is ​​used to extract the texture feature vector of the human target by calculating the gray-level co-occurrence matrix of the human body area, which is used to identify whether the human target is not wearing reflective clothing; including: converting the segmented human body area image into a grayscale image. Assume that the original human body area image is I rgb , its size is W×H. rgb Convert to grayscale image I gray , the conversion formula is:

[0050] I gray =0.299×I rgb [:,:,0]+0.587×I rgb [:,:,1]+0.114×I rgb [:,:,2], where

[0051] Irgb [:,:,0],I rgb [:,:,1],I rgb [:,:,2] represent the red, green and blue color channels of the image respectively. gray Quantization is performed. The quantization process can be regarded as dividing the grayscale value into equal intervals and mapping the original grayscale value to one of L intervals. Assume that the value range of the original grayscale value is [0, 255], and the grayscale value after quantization Q gray It can be calculated by the following formula: Among them, round represents the rounding function. The quantized gray value Q gray The value range is [0, L-1].

[0052] After quantization, the grayscale image Q gray In the example, select the target pixel (x, y), set a rectangular local window W of size N×N with the pixel as the center. Extract the pixel grayscale value in the local window W to form an N×N grayscale matrix Calculate the grayscale co-occurrence matrix P(i, j, d, θ) within the local window W. The grayscale co-occurrence matrix describes the probability that the grayscale values ​​of two pixels at a distance d in a certain direction θ are i and j respectively. In this embodiment, the direction θ includes four directions: horizontal (0°), vertical (90°), 45° and 135°, and the distance d is usually 1. Specifically, for the direction θ and the distance d, the calculation formula of the grayscale co-occurrence matrix P(i, j, d, θ) is: P(i, j, d, θ) = (the number of pixel pairs that meet the conditions in W_gray) / (the total number of pixel pairs in W_gray), where the pixel pairs that meet the conditions refer to the grayscale values ​​of two pixel points at a distance d in the direction θ, i and j respectively. The total number of pixel pairs is (Nd)×(Nd).

[0053] According to the gray level co-occurrence matrix P (i, j, d, θ), the local texture features of the human target are calculated to obtain a feature vector reflecting the texture attributes. This embodiment selects three features: contrast C, energy ASM and entropy ENT. Contrast C reflects the clarity of the image and the roughness of the texture. The calculation formula is: Among them, i and j are the row and column numbers of the gray level co-occurrence matrix, respectively, with a value range of 0 to L-1; k is the contrast coefficient, which controls the sensitivity of the contrast, with a value range of 1.5 to 2.5. The energy ASM reflects the uniformity of the gray distribution of the image and the regularity of the texture. The calculation formula is: Among them, r is the energy coefficient, which controls the rate of change of the energy value, and its value range is 1 to 2. Entropy ENT reflects the complexity of the image and the randomness of the texture. The calculation formula is: Where L is the grayscale level, ranging from 16 to 256; log is the natural logarithm function. When P(i, j) is 0, 0·log0=0 is defined. Finally, the contrast C, energy ASM and entropy ENT in the four directions are combined into a 12-dimensional texture feature vector T feat :

[0054] T feat =[C0,ASM0,ENT0,C 45 ,ASM 45 ,ENT 45 ,C 90 ,ASM 90 ,ENT 90 ,C 135 ,ASM 135 ,ENT 135 ], where the subscript represents the direction, for example, C0 represents the contrast in the horizontal direction. The grid texture features of the reflective clothing can be effectively extracted by using the texture feature extraction algorithm based on the gray level co-occurrence matrix.

[0055] The edge direction histogram algorithm is used to obtain the feature vector reflecting the contour shape of the human target by performing gradient direction statistics on the edge pixels of the human body area, which is used to identify whether the human target is not wearing a seat belt; including: body Perform Gaussian filtering. Gaussian filtering is a commonly used image smoothing technique that can suppress high-frequency noise in an image and make the edges smoother and more continuous. Assume that the standard deviation of the Gaussian filter is σ and the size of the filter is (2k+1)×(2k+1), where Represents the rounding function. Image I after Gaussian filtering smooth It can be calculated by the following formula: Among them, G(i,j) is the weight coefficient of the Gaussian filter at the (i,j) position, and the calculation formula is:

[0056] The value range of i and j is from -k to k. The value range of σ is from 0.5 to 2, which is adjusted appropriately according to the resolution and noise level of the image.

[0057] The Canny algorithm is used to filter the human body area image I smooth Perform edge detection to obtain the edge image I of the human target edge For image I smooth Calculate the first-order partial derivatives in the x and y directions respectively to obtain the gradient image I gradx and I grady. Partial derivatives can be calculated using the Sobel operator, Prewitt operator, etc. The gradient amplitude Mag and gradient direction Ang of each pixel are calculated based on the gradient image. The formula is as follows:

[0058] The gradient direction Ang is discretized and quantized into four directions: 0°, 45°, 90°, and 135°. Non-maximum suppression is performed on the gradient amplitude image Mag to find the maximum point in the local gradient direction as the candidate edge point. The candidate edge points are screened and connected using the double threshold method to obtain the final edge image I edge . Low threshold T low and high threshold T high Meet T high 2 to 3 times of T low , whose value ranges are 10% to 20% and 30% to 50% of the gradient amplitude Mag respectively.

[0059] For edge image I edge Perform morphological closing operation to fill the small holes in the edge and make the edge more continuous and complete. Let the structural element of the morphological operation be B. Usually a rectangular or circular structural element with a size of 3×3 or 5×5 is selected. Edge image I after closing operation close It can be calculated by the following formula:

[0060] in, and Θ represent morphological dilation and erosion operations respectively. The edge image I after the closing operation is processed by Sobel operator close Perform gradient calculation to obtain the gradient magnitude Mag(x,y) and gradient direction Ang(x,y) of each edge pixel. The Sobel operator is a commonly used gradient operator that performs weighted differences on the image in the x and y directions to obtain the horizontal component of the gradient G x and the vertical component G y The calculation formulas for the gradient amplitude Mag and gradient direction Ang are:

[0061] Among them, G x and G y It can be calculated by the following convolution operation: The gradient direction Ang(x,y) is divided into M intervals, and the sum of the gradient amplitudes in each interval is counted to obtain the M-dimensional gradient direction histogram Hist. Assume that the angle range of the mth interval is [θ m ,θ m+1 ), where m = 0, 1, ..., M-1. Then the mth component of the gradient direction histogram Hist(m) can be calculated by the following formula: Among them, | represents the conditional operator. When the condition is met, the value of Mag(x,y) is taken, otherwise it is 0. The value of M should be greater than 6 to ensure sufficient resolution of the gradient direction. Common values ​​are 8, 12, 16, etc.

[0062] According to the prior information of seat belt detection, the seat belt shape template Temp is constructed. The seat belt usually consists of two diagonal lines, one from the left shoulder to the right waist, and the other from the right shoulder to the left waist. These two diagonal lines present a certain angle in the image. The two diagonal lines are parallel, and the angle between them is close to 180°. The width of the diagonal line is relatively small relative to the size of the human body area and can be approximated as a line segment. Based on the above prior information, we can abstract the shape of the seat belt into two gradient direction intervals with an angle difference of 180°. Specifically, first, determine the angle range of the seat belt diagonal line. By observing a large number of sample images of seat belts being worn, we found that the angle of the seat belt diagonal line usually falls within the following range: The diagonal angle from the left shoulder to the right waist 30° to 60°; diagonal angle from right shoulder to left waist 210° to 240°; the difference between these two angle ranges is 180°, which is consistent with the characteristics of the seat belt being parallel to the diagonal line. We can select a representative angle from these two ranges as the parameter of the seat belt template, for example and or and Then, the continuous angle space is discretized into M intervals. In the previous step, we have divided the gradient direction Ang into M intervals, each interval corresponding to an angle range. Let the angle range of the mth interval be [θ m ,θ m+1 ), where m = 0, 1, ....., M-1. The intervals should be divided as evenly as possible to ensure that the angle span of each interval is equal, that is: θ m+1 -θ m =360° / M. For example, when M=8, the angle space is divided into 8 intervals, each with a span of 45°. and Determine the gradient direction interval to which they belong. We can calculate its corresponding interval number m1 by the following formula: Among them, round represents the rounding function. (360° / M) represents the angle span of each interval. For example, when M=8, , m1 = round(45° / 45°) = 1, indicating that the angle 45° falls within the first interval. We can calculate its corresponding interval number m2: For example, when M=8, When , m2 = round (225° / 45°) = 5, which means that the angle 225° falls within the fifth interval. Construct the seat belt shape template Temp. The template Temp is a vector with a length of M, and each element corresponds to a gradient direction interval. We will and The corresponding interval is set to 1, and the remaining intervals are set to 0, that is: Among them, m=0,1,....,M-1. In this way, we get a binary template with values ​​of 1 only at two angle positions and values ​​of 0 at other positions. In this embodiment, M=8, Then the seat belt shape template Temp can be expressed as: Temp = [0, 1, 0, 0, 0, 1, 0, 0]. This template vector indicates that the value is 1 in the first interval (corresponding to 45°) and the fifth interval (corresponding to 225°), and the value is 0 in the remaining intervals.

[0063] Calculate the similarity Sim between the gradient direction histogram Hist and the seat belt shape template Temp to obtain the seat belt detection similarity map Sim map The similarity calculation uses the normalized correlation coefficient formula:

[0064] Among them, Hist(x,y,m) represents the mth component of the gradient direction histogram in the local area centered on the pixel (x,y). mean and Temp mean They are the means of the histogram Hist and the template Temp respectively.

[0065] In order to improve the robustness and position invariance of detection, when calculating the similarity, it is necessary to perform a sliding window operation on the seat belt shape template Temp to obtain the similarity values ​​at different positions and scales. Assume that the sliding step size of the template is (step x ,step y ), the sliding range is (range x ,range y ), then the similarity calculation formula can be expanded to:

[0066] ; where u and v are the offsets of the template in the x and y directions, respectively, and their value range is [0,range x / step x ]and ε is a small positive number used to avoid the denominator being zero, and its value range is 0.001 to 0.01. x ,range y) should be reasonably set according to the size and position of the seat belt in the image, usually about half of the width and height of the human body area. Finally, according to the seat belt detection similarity map Sim map , to determine whether the human target is not wearing a seat belt. A similarity threshold T can be set sim , when Sim map The maximum similarity value in is less than T sim When , it is considered that the seat belt is not detected, that is, the human target is not wearing the seat belt; otherwise, it is considered that the seat belt is detected, that is, the human target is wearing the seat belt. sim The value of needs to be debugged and optimized according to actual data, and is usually between 0.6 and 0.8.

[0067] The extracted color feature vector, texture feature vector and shape feature vector are combined as image features. The dimension of the color feature vector is D color , the dimension of the texture feature vector is D texture , the dimension of the shape feature vector is D shape . Normalize each eigenvector. Since the numerical ranges of different eigenvectors may vary greatly, in order to balance the contribution of each feature, the eigenvectors need to be normalized. The normalized eigenvectors are concatenated into a combined eigenvector. Let the combined eigenvector be f combined ,but:

[0068] Where [·] represents a vector concatenation operation. combined The dimension is D combined =D color +D texture +D shape .

[0069] S3, based on the image features, build a convolutional neural network risk control model based on the attention mechanism, including: representing the input image I as a H×W×C tensor, where H, W, and C are the height, width, and number of channels of the image respectively. Input the image into the convolutional neural network, and after L convolutional layers and M pooling layers, obtain the semantic feature map F of the image. The parameters of the lth convolutional layer are: the number of convolution kernels K l , convolution kernel size F l ×F l , step length S l and fill P l ; The parameters of the mth pooling layer are: pooling type (maximum pooling or average pooling), pooling kernel size R m ×R m and step length S mThe size of the semantic feature map F is H'×W'×D, where D is the number of convolution kernels in the last convolution layer. Flatten F into a D-dimensional semantic feature vector f semantic .

[0070] An attention layer is set at the last layer of the convolutional neural network to use the attention mechanism to analyze the semantic feature vector f semantic Perform weighted fusion. The attention layer contains A attention heads, each of which corresponds to a weight matrix W a , the size is D×(D / A). semantic Get A D / A-dimensional attention feature vectors through the attention head a=1,2,.....,A,f att =[f att1 ,f att2 ,......,f attA ], and then concatenate them into a D-dimensional enhanced semantic feature vector f att . The semantic feature vector f att Input a fully connected layer and map it into a Q-dimensional probability distribution vector of hazard source categories through a softmax activation function. Where Q is the number of hazardous source categories. The weight matrix W of the fully connected layer fc The size is D×Q, and the bias vector b fc The size is Q. The softmax function normalizes the category probability to the range of (0, 1) so that the sum of the probabilities is 1. The output of the fully connected layer is: z = f att ×W fc +b fc .

[0071] S4, the coding rules of the hazard category label are: Category 0: normal (wearing a seat belt and a reflective vest), Category 1: not wearing a seat belt, Category 2: not wearing a reflective vest, Category 3: not wearing a seat belt and not wearing a reflective vest. The category probability distribution p of the model output clsis a 4-dimensional vector, each element represents the probability that the image belongs to the corresponding category: p_cls = [p0, p1, p2, p3], where p0, p1, p2, p3 are the probability values ​​of categories 0, 1, 2, and 3, respectively, satisfying p0+p1+p2+p3=1. Set a threshold τ (such as 0.5), and when the probability value of a category is greater than τ, the image is judged to belong to that category. Since an image may contain multiple sources of danger at the same time, it is necessary to judge each category separately. Determine whether a seat belt is not worn: If p1>τ or p3>τ, it is considered that there is a situation where a seat belt is not worn in the image, and the output result is "no seat belt is worn". Otherwise, it is considered that there is no situation where a seat belt is not worn in the image. Determine whether a reflective vest is not worn: If p2>τ or p3>τ, it is considered that there is a situation where a reflective vest is not worn in the image, and the output result is "no reflective vest is worn". Otherwise, it is considered that there is no situation where a reflective vest is not worn in the image.

[0072] After determining that there is no safety belt or reflective clothing in the image, the location coordinates of the danger source need to be output. The location coordinates output by the location regression branch of the model are: t' = (x', y', w', h'), where (x', y') are the predicted center coordinates of the danger source, and w' and h' are the predicted width and height of the danger source. The predicted coordinates are restored to the coordinates in the image coordinate system to obtain the detection frame of the danger source: The coordinates of the detection box are (x1', y1', x2', y2'), which indicates the location of the danger source in the image.

[0073] Construct a loss function, optimize the parameters of the convolutional neural network through back propagation, minimize the loss function, and obtain the trained risk control model. The loss function consists of three parts: Category recognition loss of not wearing reflective clothing Loss clswl , Classification loss of not wearing seat belts Loss clsab and the hazard source location regression loss Loss reg For category recognition loss Loss clswl and Loss clsab , using the weighted cross entropy loss function:

[0074]

[0075] Among them, N is the number of training samples, y 0i and 1i They represent the true label (values ​​are 0 or 1) of whether sample i is wearing reflective clothing or not, respectively, and p 0i and p 1i The model predicts the probability of sample i wearing reflective clothing and not wearing reflective clothing; y 2i and3i They represent the true label of whether sample i is wearing a seat belt or not, p 2i and p 3i The model predicts the probability of sample i wearing a seat belt or not wearing a seat belt; w0, w1, w2, w3 are the balance weights of the four types of samples, which are used to alleviate the problem of class imbalance.

[0076] For position regression loss Loss reg , using CIOU loss function: Among them, IOU i is the prediction box t i 'With the real frame i The intersection-and-union ratio, Dist(t i -t i ') is the Euclidean distance between the center of the predicted box and the center of the real box, c is the diagonal distance of the minimum closure area covering both the predicted box and the real box, is the consistency measure of the predicted box aspect ratio and the real box aspect ratio, and α is the trade-off factor that balances the IOU term and the aspect ratio consistency term. The three loss functions are weighted and summed to obtain the total loss function Loss:

[0077] Loss = α × Loss clswl +β×Loss clsab +γ×Loss reg , where α, β, and γ are balance factors that control the contribution ratio of the three losses to the total loss. During the model training process, the weight parameters of the convolutional neural network are continuously updated iteratively through optimization algorithms such as stochastic gradient descent to minimize the loss function Loss. When the loss function converges to a smaller value, the final risk control model is obtained.

[0078] like Figure 3 As shown in the figure, the hazard source identification result is transmitted to the edge computing node, and a hazard source distribution map of the construction site is generated according to the category and location of the hazard source, and the hazard source management of the construction site is carried out according to the hazard source distribution map. Specifically, the risk control model identifies a total of N hazard sources within the time period T, and the information of each hazard source is represented by a tuple (t, m, c, x, y, w, h), where t is the timestamp of the hazard source identification, m is the area number where the hazard source is located, c is the category of the hazard source (0 means not wearing reflective clothing, 1 means not wearing a safety belt, 2 means not wearing reflective clothing and not wearing a safety belt at the same time), and x, y, w, and h are the center coordinates and width and height of the hazard source, respectively.

[0079] The N tuples are packaged into a list D and transmitted to the edge computing node via the wireless network. The transmission bandwidth is BMbps, the data packet size is SMB, and the transmission delay is about S / B seconds. In the edge computing node, list D is grouped according to the category c of the hazard source. Each tuple in the list is traversed and added to the corresponding group according to its category c: if c = 0, the tuple is added to the group G1 of not wearing reflective clothing; if c = 1, the tuple is added to the group G2 of not wearing safety belts; if c = 2, the tuple is added to the group G3 of not wearing reflective clothing and not wearing safety belts at the same time. After the grouping is completed, three sublists G1, G2 and G3 are obtained, which contain all the hazard source information of not wearing reflective clothing, not wearing safety belts, and not wearing reflective clothing and not wearing safety belts at the same time. The tuple format in the sublist is consistent with D.

[0080] On the construction site plan P, the hazardous sources are marked according to the hazardous source location information in G1, G2 and G3. The plan P can be regarded as a two-dimensional matrix of size W×H. Each element corresponds to a location on the site. The initial value is 0, indicating safety. Traverse each tuple (t, m, c, x, y, w, h) in G1 and draw a red triangle mark at the corresponding position of the plan P. The side length of the triangle is L, and the coordinates of the three vertices of the triangle are: Traverse each tuple (t, m, c, x, y, w, h) in G2, and draw a yellow circle at the corresponding position of the plane graph P. The radius of the circle is R, and the coordinates of the center of the circle are (x, y). Fill the area with yellow with the center of the circle and R as the radius. Traverse each tuple (t, m, c, x, y, w, h) in G3, and draw a red triangle and a yellow circle at the corresponding position of the plane graph P, indicating that the person is not wearing a reflective vest and a safety belt at the same time.

[0081] After the labeling is completed, a construction site plan P' is obtained with a red triangle mark (indicating that reflective clothing is not worn), a yellow circle mark (indicating that a safety belt is not worn), and a red and yellow combination mark (indicating that both reflective clothing and safety belts are not worn). Save the labeled plan P' as a hazard source distribution map I. The image format can be selected in common formats such as PNG and JPEG. The resolution of the image is M×N, and each pixel occupies 3 bytes (RGB format), so the image size is approximately 3×M×N bytes. Save the hazard source distribution map I to the local storage of the edge computing node. The file name can contain a timestamp T, such as "danger_map_T.png".

[0082] According to the hazard source distribution map I, the safety hazards at the construction site are managed and dealt with. Specific measures include: for the hazard source of not wearing reflective clothing (red triangle mark), notify the site management personnel of the corresponding area and require the construction personnel to wear reflective clothing immediately; for the hazard source of not wearing safety belts (yellow circle mark), notify the site management personnel of the corresponding area and require the construction personnel to wear safety belts immediately; for the hazard source of not wearing reflective clothing and safety belts at the same time (red and yellow combination mark), give priority to notifying the site management personnel of the corresponding area, require the construction personnel to wear reflective clothing and safety belts immediately, and strengthen the safety inspection and management of the area; for areas with repeated hazard sources, strengthen safety training and on-site inspections to avoid the recurrence of dangerous situations; regularly generate and update the hazard source distribution map, track the changes in the safety status of the construction site, and promptly discover and eliminate new safety hazards.

Claims

1. A smart construction site hazard source management method, characterized in that: include: Collect construction site images; The collected images are transmitted to the edge computing node, the images are preprocessed, and image features are extracted. The image features include the color, texture, and shape of the target; According to the image features, a convolutional neural network risk control model based on the attention mechanism is constructed; The risk control model is used to identify the hazard sources in each area. The hazard source identification results include the category and location of the hazard source; the hazard source categories include not wearing a safety belt and not wearing a reflective vest; The hazard source identification results are transmitted to the edge computing node, and a hazard source distribution map of the construction site is generated according to the category and location of the hazard source. The hazard source management of the construction site is carried out according to the hazard source distribution map; The collected images are transmitted to the edge computing node and pre-processed, including: The semantic-based image segmentation algorithm is used to semantically annotate the pixels in the image and segment the image into the human body area and the background area. The target detection algorithm based on the region proposal network is used to detect human targets in the image by generating candidate regions and performing classification and regression. The HSV color space statistical algorithm is used to obtain the characteristic vector reflecting the color distribution of the human body by performing histogram statistics on the human body area. The texture feature extraction algorithm based on gray-level co-occurrence matrix is ​​used to extract the texture feature vector of the human target by calculating the gray-level co-occurrence matrix of the human body area, which is used to identify whether the human target is not wearing reflective clothing; The edge direction histogram algorithm is used to obtain the feature vector reflecting the contour shape of the human target by performing gradient direction statistics on the edge pixels of the human body area, which is used to identify whether the human target is not wearing a seat belt. The extracted color feature vector, texture feature vector and shape feature vector are combined as image features.

2. The smart construction site hazard source management method according to claim 1 is characterized by: Segment the image into human body area and background area, including: A pre-trained semantic segmentation network is used to extract features from images. The semantic segmentation network adopts an encoder-decoder structure. The encoder extracts image features through convolution and downsampling, and the decoder restores the resolution of the feature map through upsampling and skip connections. A fully convolutional network is used to map the extracted features into pixel-level category probability maps; According to the pixel-level category probability map, the threshold segmentation method is used to segment the image into the human body area and the background area. The human body area corresponds to the area of ​​human body pixels in the category probability map, and the background area corresponds to the area of ​​non-human body pixels in the category probability map.

3. The smart construction site hazard source management method according to claim 2 is characterized by: Extract the texture feature vector of the human target, including: The segmented human body region is converted into a grayscale image, and the grayscale image is quantized according to a preset grayscale level L to obtain a quantized grayscale image; In the quantized grayscale image, a target pixel is selected, a rectangular local window is set with the target pixel as the center, and the pixel grayscale value of the local window is extracted; Calculate the gray level co-occurrence matrix P(i,j,d,θ) in the local window, where P(i,j,d,θ) represents the probability that the gray levels of two pixels with a distance d in the direction θ are i and j respectively; the direction θ includes four directions: horizontal, vertical, 45 degrees and 135 degrees; The texture features are calculated according to the gray-level co-occurrence matrix to obtain the feature vector reflecting the local texture attributes of the human target. The texture features include contrast C, energy ASM and entropy ENT.

4. The smart construction site hazard source management method according to claim 3 is characterized by: The contrast ratio C is calculated by the following formula: Among them, k is the contrast coefficient, ranging from 1.5 to 2.5; Energy ASM is calculated by the following formula: Among them, r is the energy coefficient, ranging from 1 to 2; Entropy ENT is calculated by the following formula: Wherein, L is the grayscale level, ranging from 16 to 256; P(i,j) is the element of the grayscale co-occurrence matrix, which indicates the probability that the grayscale values ​​of two pixels with a distance of preset step size d in the preset direction θ are i and j respectively; i and j are the row and column numbers of the grayscale co-occurrence matrix, ranging from 0 to L-1, indicating the quantized pixel grayscale value.

5. The smart construction site hazard source management method according to claim 2 is characterized by: Obtain the feature vector reflecting the contour shape of the human target, including: Perform Gaussian filtering on the segmented human body area image; The Canny algorithm is used to perform edge detection on the filtered human body area image to obtain the edge image of the human body target; Perform morphological closing operation on the edge image; The Sobel operator is used to calculate the gradient of the edge image after the closed operation to obtain the gradient amplitude Mag(x,y) and gradient direction Ang(x,y) of each edge pixel; Divide the gradient direction Ang(x,y) into M intervals, count the sum of the gradient amplitudes in each interval, and obtain an M-dimensional gradient direction histogram, where the value of M is greater than or equal to 4; According to the seat belt detection prior information, a seat belt shape template is constructed, and the seat belt shape template uses two parallel diagonal lines; Calculate the similarity Sim(x,y) between the gradient direction histogram and the seat belt shape template to obtain the seat belt detection similarity map; According to the seat belt detection similarity graph, it is judged whether the human target is not wearing a seat belt.

6. The smart construction site hazard source management method according to claim 5 is characterized by: Calculate the similarity Sim(x,y) between the gradient direction histogram and the seat belt shape template using the following formula: Among them, Hist(x,y) is the value of the gradient direction histogram at the position (x,y), which represents the accumulated amplitude of the edge pixel points in the local area centered at (x,y) in the corresponding gradient direction interval; Temp(xu,yv) is the value of the seat belt shape template at the position (xu,yv), which represents the value of the corresponding position after the template is offset by (u,v) units relative to (x,y), reflecting the gradient direction distribution of the edge of the seat belt; Hist mean and Temp mean are the means of Hist and Temp respectively; ε is a small positive constant used to avoid the denominator being zero, and its value range is 0.001 to 0.01; u and v are the offsets of the template in the x and y directions respectively, and their value range is 0 to the image width / height minus the template width / height.

7. The smart construction site hazard source management method according to any one of claims 2 to 6, characterized in that: Construct a convolutional neural network risk control model based on the attention mechanism, including: Input the image features into the convolutional neural network, and obtain the semantic features of the image through convolution and pooling operations; An attention layer is set at the last layer of the convolutional neural network, and the semantic features are weighted and fused through the attention mechanism to obtain the semantic feature vector after attention enhancement. The semantic feature vector after attention enhancement is input into the fully connected layer, and the semantic feature vector is mapped into the category probability distribution of the hazard source through the softmax activation function; According to the probability distribution of the category of the hazard source, determine whether there is a person not wearing a safety belt or a reflective vest in the image. If so, output the category and location coordinates of the hazard source; Construct a loss function, optimize the parameters of the convolutional neural network through back propagation, minimize the loss function, and obtain the trained risk control model.

8. The smart construction site hazard source management method according to claim 7 is characterized by: The loss function expression is as follows: Loss=α×Loss clswl +β×Loss clsab +γ×Loss reg Among them, Loss is the total loss function, Loss clswl Loss for the category identification of not wearing reflective clothing, clsab Identify the loss for the category of not wearing seat belts, Loss reg is the regression loss of the hazard source location; α, β and γ are balance factors, which are used to adjust the contribution weights of the three losses to the total loss, and their values ​​range from 0 to 1; Among them, N is the number of training samples, y 0i Indicates the true label of whether sample i wears reflective clothing, with a value of 0 or 1, y 1i Indicates the true label of whether sample i is not wearing reflective clothing, with a value of 0 or 1, and y 0i +y 1i =1; p 0i The probability that the risk control model predicts that sample i wears reflective clothing, p 1i is the probability that the risk control model predicts that sample i does not wear reflective clothing, and p 0i +p 1i =1; w0 and w1 are the balance weights of the two types of samples; Among them, y 2i Indicates the true label of whether sample i wears a seat belt, y 3i The true label indicating whether sample i is not wearing a seat belt; p 2i is the probability value of sample i wearing a seat belt predicted by the risk control model, p 3i is the probability value predicted by the risk control model that sample i does not wear a seat belt; w2 and w3 are the weight coefficients of the two types of samples; Among them, N is the number of training samples, IOU i is the prediction box t i 'With the real frame i The intersection-and-union ratio, Dist(t i -t i ') is the Euclidean distance between the center of the predicted box and the center of the real box; c is the diagonal distance of the minimum closure area covering both the predicted box and the real box; It represents the consistency measure between the aspect ratio of the predicted box and the aspect ratio of the real box; α is a trade-off factor used to balance the contribution of the IOU term and the aspect ratio consistency term to the loss, and its value range is 0 to 1.

9. The smart construction site hazard source management method according to claim 8 is characterized by: Management of construction site hazards is carried out according to the hazard distribution map, including: Transmit the categories and location coordinates of the danger sources identified by the risk control model to the edge computing nodes; In the edge computing node, the hazard sources are grouped according to their categories. The hazard sources without reflective clothing are divided into the first group, and the hazard sources without safety belts are divided into the second group. The hazard sources without both reflective clothing and safety belts are divided into the third group. According to the first group of hazards, red signs are marked at the corresponding positions in the construction site plan. The red signs are in the form of triangles, indicating the hazard sources where reflective clothing is not worn; According to the second group of danger sources, yellow signs are marked at the corresponding positions in the construction site plan. The yellow signs are circular and indicate the danger sources of not wearing safety belts; According to the third group of hazards, red and yellow signs are marked at the corresponding locations in the construction site plan; The construction site plan marked with hazard identification will be used as the hazard distribution map.

10. A smart construction site hazard source management system, characterized in that: include: At least one processing unit; used to execute instructions to implement the smart construction site hazard source management method as described in any one of claims 1 to 9.

Images