Building construction operation monitoring and early warning system based on image processing

Abstract

The invention discloses a building construction operation monitoring and early warning system based on image processing, and relates to the technical field of image data processing. The building construction operation monitoring and early warning system based on image processing comprises an identification and analysis unit which is used for acquiring a high-risk area image and judging whether an operator exists; the operation image acquisition unit is used for continuously acquiring images after the personnel are detected; the operation image analysis unit is used for extracting key points in combination with a feature extraction model, and calculating an action violence index, an attitude anomaly index and an operation instability index; the operation risk analysis unit is used for comprehensively calculating a risk index and comparing the risk index with a preset threshold value; and the operation risk early warning unit gives out early warning when the risk is abnormal, and improves the operation safety response capability. According to the invention, a YOLOv5 model is adopted to carry out instance-level identification on operators, so that the system adapts to a complex environment, the identification precision is improved, and accurate input is provided for subsequent risk analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a construction operation monitoring and early warning system based on image processing. Background Art

[0002] At construction sites, worker safety has always been a core issue in project management. Due to the complex construction environment, narrow space, and dense machinery and equipment, workers are prone to safety accidents when working at heights, working near edges, and cross-operating. Traditional construction safety management methods rely heavily on manual inspections, supervisory oversight, or manual playback of fixed camera images. These methods make it difficult to detect abnormal behavior or dangerous conditions in a timely manner, and they suffer from problems such as large lags, high recognition errors, and high labor costs.

[0003] Prior art, such as the invention patent application with publication number CN116402863B, discloses an intelligent analysis and early warning system for construction monitoring data. The system acquires a grayscale image for each frame in a video; acquires the optical flow of pixels based on adjacent grayscale images; divides the optical flow into directional intervals according to their direction, and acquires characteristic optical flow based on the optical flow in the directional intervals; sorts the characteristic optical flow to acquire a motion feature sequence for a sliding window region; acquires a motion abrupt change coefficient for the sliding window region based on the motion feature sequences of adjacent sliding window regions to determine the motion region; acquires the change entropy of the grayscale image based on the motion abrupt change coefficient of the sliding window region and the distribution index of the directional intervals within the motion region; acquires the temporal motion change entropy based on the change entropy of continuous grayscale images; and determines motion anomalies in the monitoring video based on the temporal motion change entropy and issues an early warning. The present invention improves the accuracy of motion anomaly detection and the timeliness of early warning by adaptively acquiring motion regions.

[0004] Based on the above solution, it is found that the limitations of the existing technology include at least the following problems. Although the existing technology realizes the detection of moving areas in the video through optical flow calculation and sliding window clustering, its processing method is based on the distribution law of pixel-level grayscale changes and optical flow direction. It is mainly suitable for judging whether there is abnormal movement in the area. It is difficult to achieve identity confirmation of specific operators, structural posture analysis and fine behavior judgment. Especially in scenarios where multiple people are working at the same time or the target is stationary but has an abnormal posture, there are problems such as misjudgment of moving areas and high missed alarm rate. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a construction operation monitoring and early warning system based on image processing, which solves the problem of difficulty in achieving accurate operation risk identification and real-time early warning based on the structured behavioral characteristics of personnel.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a construction operation monitoring and early warning system based on image processing, comprising: an identification and analysis unit, configured to acquire regional image data of high-risk areas of a construction site in real time, and determine whether there are workers in the high-risk areas of the construction site based on a pre-trained target detection model; an operation image acquisition unit, configured to continuously acquire operation image data of the workers at several time points when there are workers in the high-risk areas of the construction site; an operation image analysis unit, configured to combine the operation image data of the workers at several time points with a pre-trained feature extraction model to analyze the operation feature set of the workers in the high-risk areas of the construction site, and analyze the operation feature index set of the workers in the high-risk areas of the construction site, wherein the operation feature index set includes a motion intensity index, an abnormal posture index, and an operation instability index; an operation risk analysis unit, configured to analyze a comprehensive operation risk index of the workers in the high-risk areas of the construction site based on the operation feature index set of the workers in the high-risk areas of the construction site, and perform judgment and analysis based on a preset operation wind range; and an operation risk early warning unit, configured to send an operation risk alert when the comprehensive operation risk index of the workers in the high-risk areas of the construction site is within the preset operation wind range.

[0007] Furthermore, the regional image data includes pixel values of a plurality of regional pixel points, and the target detection model is specifically a YOLOv5 model, and the YOLOv5 model includes an image feature extraction subnetwork, a feature fusion subnetwork, and a target detection head subnetwork.

[0008] Furthermore, based on the pre-trained target detection model, the specific steps for determining whether there are workers in the high-risk areas of the construction site are as follows: in the image feature extraction subnetwork of the YOLOv5 model, the regional image data of the high-risk areas of the construction site is received, and normalization and resizing are performed to generate a standardized image tensor, and multi-level convolution operations and downsampling are performed to extract the spatial texture features and edge gradient features in the construction site image to obtain a multi-scale semantic feature map of the construction image; in the feature fusion subnetwork of the YOLOv5 model, the multi-scale semantic feature map of the construction image is subjected to cross-layer connection processing and feature channel splicing processing to fuse the high-resolution structural features and deep semantic information in the image to obtain a fused multi-level feature map; in the target detection head subnetwork of the YOLOv5 model, the fused multi-level feature map is subjected to convolution decoding processing, and bounding box regression processing and category confidence scoring processing are performed at different detection scales to obtain the detection box set, confidence value and category identification of each candidate target in the construction site image, and determine whether there are workers in the high-risk areas of the construction site.

[0009] Furthermore, the operation image data includes pixel values and two-dimensional coordinates of several pixel points, and the feature extraction model is specifically a human posture estimation model, which includes an image feature extraction layer, a posture encoding layer, and a key point prediction head.

[0010] Furthermore, the specific steps for analyzing the work feature set of workers in high-risk areas of the construction site are as follows: the work image data of the workers at several time points are respectively input into the pre-trained human posture estimation model for feature extraction processing to obtain the two-dimensional coordinates and confidence of several key points of the workers at each time point; based on the two-dimensional coordinates and confidence of several key points of the workers at each time point, the work feature set of workers in high-risk areas of the construction site is analyzed, and the work feature set includes motion amplitude, posture deviation angle, joint movement frequency, vertical displacement change rate, number of motion mutations, posture symmetry index, confidence mean, body center of gravity trajectory change, and joint point jitter index.

[0011] Furthermore, the specific steps for obtaining the two-dimensional coordinates and confidence levels of several key points of the operator at each time point are as follows: in the image feature extraction layer of the human posture estimation model, the work image data of the operator at several time points are received, and the image data are respectively normalized and resized to generate a standardized image tensor, and the standardized image tensor is subjected to multi-level convolution operations, downsampling processing, and nonlinear activation processing to extract the underlying texture features, edge response features, and spatial structure features of the work image to obtain a multi-scale semantic feature map; in the posture encoding layer of the human posture estimation model, the multi-scale semantic feature map is subjected to feature fusion processing and structural relationship modeling processing, and the spatial connection relationship between the key parts of the operator's body is constructed to obtain a structural expression map containing the semantic information of the operator's whole-body skeletal structure; in the key point prediction head of the human posture estimation model, the structural expression map is subjected to multi-channel convolution prediction processing and probability normalization processing, and a key point heat map and a confidence map are generated in each key point channel respectively, and the key point position coordinates are analyzed based on the heat map response value to output the two-dimensional coordinates and confidence levels of several key points of the operator at each time point.

[0012] Furthermore, the specific formula for calculating the comprehensive operation risk index of workers in high-risk areas of the construction site is as follows: Among them, ZyF is the comprehensive operation risk index of the workers in the high-risk area of the construction site, DzJ is the movement intensity index of the workers in the high-risk area of the construction site, λ1 is the movement intensity influence coefficient stored in the database, ZtY is the posture abnormality index of the workers in the high-risk area of the construction site, λ2 is the posture abnormality influence coefficient stored in the database, CzS is the operation instability index of the workers in the high-risk area of the construction site, λ3 is the operation instability influence coefficient stored in the database, DzF is the movement amplitude of the workers in the high-risk area of the construction site, and ZtP is the posture deviation angle of the workers in the high-risk area of the construction site.

[0013] Furthermore, the specific steps for analyzing the set of operation characteristic indexes of workers in high-risk areas of the construction site are as follows: based on the motion amplitude, joint movement frequency, and confidence mean of the workers in high-risk areas of the construction site, analyzing the motion intensity index of the workers in high-risk areas of the construction site; based on the posture deviation angle, number of motion mutations, posture symmetry index, and confidence mean of the workers in high-risk areas of the construction site, analyzing the posture abnormality index of the workers in high-risk areas of the construction site; based on the vertical displacement change rate, body center of gravity trajectory change, and joint point jitter index of the workers in high-risk areas of the construction site, analyzing the operation instability index of the workers in high-risk areas of the construction site.

[0014] Furthermore, the specific formula for calculating the intensity index of workers in high-risk areas of the construction site is as follows: DzJ = ln(1+DzF*DzP)*(1+e -ZxJ ); where DzJ is the intensity index of the workers in the high-risk area of the construction site, DzF is the amplitude of the workers in the high-risk area of the construction site, DzP is the joint movement frequency of the workers in the high-risk area of the construction site, e is a natural constant, and ZxJ is the mean confidence level of the workers in the high-risk area of the construction site.

[0015] Furthermore, the specific formula for calculating the abnormal posture index of workers in high-risk areas of the construction site is as follows: Among them, ZtY is the posture abnormality index of the workers in the high-risk area of the construction site, ZtP is the posture deviation angle of the workers in the high-risk area of the construction site, DzT is the posture symmetry index of the workers in the high-risk area of the construction site, DzT is the number of sudden changes in the movements of the workers in the high-risk area of the construction site, ZxJ is the mean confidence value of the workers in the high-risk area of the construction site, and δ is the confidence adjustment factor stored in the database.

[0016] The present invention has the following beneficial effects:

[0017] (1) The construction operation monitoring and early warning system based on image processing sets up a recognition and analysis unit and uses the pre-trained YOLOv5 to perform deep recognition and analysis on the image data of high-risk areas in the construction site. It can effectively determine whether there are workers in the area. Unlike the existing technology that mainly relies on indirect pixel change methods such as optical flow field changes or background modeling, the system is based on feature convolution and target detection mechanisms, can directly identify instances of workers, and has bounding box positioning and confidence judgment functions. It is suitable for common construction site scenes such as multi-person operations, complex backgrounds, and non-continuous target movement. It solves the problems of unstable recognition and high misjudgment rate of traditional methods when the target is stationary, occluded or under complex lighting conditions. At the same time, the YOLOv5 model has good small target recognition capabilities and real-time processing efficiency through the synergy of image feature extraction sub-network, feature fusion sub-network and detection head sub-network, ensuring that the system can immediately trigger the subsequent image analysis process when operations in high-risk areas begin, providing an accurate target basis for subsequent risk assessment.

[0018] (2) The construction operation monitoring and early warning system based on image processing obtains the coordinates and confidence of the key points of the operator's body through key point prediction, and extracts nine types of operation behavior characteristic parameters such as movement amplitude, posture deviation angle, joint movement frequency, and vertical displacement change rate to form a structured operation feature set. Based on this feature set, it further constructs the movement intensity index, posture abnormality index and operation instability index, and models and identifies potential risks such as rapid arm swinging, posture deviation, and sliding tendency, thereby overcoming the limitation of the existing technology that can only judge whether there is movement but is difficult to identify the degree of movement abnormality. It introduces quantitative standards for construction safety supervision, has good interpretability and parameter tracking, and provides solid support for early warning of high-risk actions.

[0019] (3) The construction operation monitoring and early warning system based on image processing constructs an operation risk analysis unit based on the operation characteristic index set, and generates a unified operation risk comprehensive index by setting the action intensity influence coefficient, posture abnormality influence coefficient and operation instability influence coefficient stored in the database. The risk influence coefficient is based on the historical data statistics and accident annotation data of the construction site, and the mapping relationship between the behavior parameters and the risk level is constructed through regression analysis, Bayesian scoring and other methods to ensure that the influence weights of each index in different operation scenarios are reasonably distributed, thereby forming a risk assessment mechanism with site adaptability. Compared with the traditional system based on fixed threshold judgment, the risk fusion mechanism introduced by this system has dynamic adjustment capabilities, which can accurately distinguish the risk degree of the same action under different working conditions, avoid false alarms and missed alarms, and automatically issue a risk warning when the risk comprehensive index exceeds the set range through the operation risk early warning unit linkage alarm mechanism, effectively improving the timeliness and automation level of construction safety response.

[0020] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a block diagram of the construction operation monitoring and early warning system based on image processing of the present invention.

[0022] Figure 2 This is a flowchart of the specific steps for determining whether there are workers in high-risk areas of a construction site in the construction operation monitoring and early warning system based on image processing of the present invention.

[0023] Figure 3 This is a flowchart of the specific steps for analyzing the operation feature set of workers in high-risk areas of a construction site in the construction operation monitoring and early warning system based on image processing of the present invention. DETAILED DESCRIPTION

[0024] See also Figure 1 An embodiment of the present invention provides a technical solution: a construction operation monitoring and early warning system based on image processing, comprising: an identification and analysis unit for acquiring regional image data of a high-risk area of a construction site in real time, and determining whether there are workers in the high-risk area of the construction site based on a pre-trained target detection model; an operation image acquisition unit for continuously acquiring operation image data of the workers at a plurality of time points when there are workers in the high-risk area of the construction site; an operation image analysis unit for combining the operation image data of the workers at the plurality of time points with a pre-trained feature extraction model to analyze an operation feature set of the workers in the high-risk area of the construction site, and analyzing an operation feature index set of the workers in the high-risk area of the construction site, the operation feature index set including a motion intensity index, an abnormal posture index, and an operation instability index; an operation risk analysis unit for analyzing a comprehensive operation risk index of the workers in the high-risk area of the construction site based on the operation feature index set of the workers in the high-risk area of the construction site, and performing a judgment and analysis based on a preset operation wind range; and an operation risk early warning unit for sending an operation risk alert to relevant personnel when the comprehensive operation risk index of the workers in the high-risk area of the construction site is within the preset operation wind range.

[0025] The specific formula for calculating the comprehensive operational risk index for workers in high-risk areas of a construction site is as follows: Among them, ZyF is the comprehensive operation risk index of the workers in the high-risk area of the construction site, DzJ is the movement intensity index of the workers in the high-risk area of the construction site, λ1 is the movement intensity influence coefficient stored in the database, ZtY is the posture abnormality index of the workers in the high-risk area of the construction site, λ2 is the posture abnormality influence coefficient stored in the database, CzS is the operation instability index of the workers in the high-risk area of the construction site, λ3 is the operation instability influence coefficient stored in the database, DzF is the movement amplitude of the workers in the high-risk area of the construction site, and ZtP is the posture deviation angle of the workers in the high-risk area of the construction site.

[0026] It should be explained that the specific steps for obtaining the action intensity influence coefficient λ1, posture abnormality influence coefficient λ2, and operation instability influence coefficient λ3 stored in the database are as follows: first, a video image database containing multiple types of construction action samples is constructed, and for each type of construction action, the corresponding key point coordinate sequence and behavior feature parameter set are collected; then, based on the historical annotation data, high-risk behavior samples with known safety accident risks are classified and labeled, and the mean and fluctuation range of the action amplitude parameters, posture offset angle parameters, and vertical displacement parameters corresponding to each type of behavior are calculated; further, the contribution of each type of behavior parameter to the past operation risk level is statistically analyzed, and the probability mapping relationship between the behavior feature parameters and the risk level is constructed using regression analysis or Bayesian scoring methods; finally, the weighted influence of each parameter on the operation risk index is extracted according to the probability mapping result, and stored as the action intensity influence coefficient λ1, posture abnormality influence coefficient λ2, and operation instability influence coefficient λ3, respectively, for subsequent calculation and call of the comprehensive operation risk index.

[0027] The specific implementation example for calculating the comprehensive operation risk index of workers in high-risk areas of a construction site is as follows, with the following parameters:

[0028] The motion intensity index of workers in high-risk areas of the construction site is approximately: 0.738.

[0029] The motion intensity coefficient stored in the database is approximately 1.260.

[0030] The abnormal posture index of workers in high-risk areas of the construction site is approximately: 0.652.

[0031] The posture abnormality influence coefficient stored in the database is approximately: 1.180.

[0032] The operational instability index of workers in high-risk areas of the construction site is approximately: 0.714.

[0033] The operational instability impact coefficient stored in the database is approximately: 1.095.

[0034] The movement amplitude of workers in high-risk areas of the construction site is approximately: 0.423.

[0035] The posture deviation angle of workers in high-risk areas of the construction site is approximately: 0.312.

[0036] Substituting the above data into the specific formula for calculating the comprehensive operation risk index of workers in high-risk areas of the construction site, we obtain:

[0037] The comprehensive operational risk index of workers in high-risk areas of the construction site = (((0.738)^1.260)+((0.652)^1.180)+((0.714)^1.095)) / (1+ln(1+0.423+0.312))≈1.252.

[0038] Specifically, the regional image data includes pixel values of several regional pixels, and the target detection model is specifically a YOLOv5 model. The YOLOv5 model includes an image feature extraction subnetwork, a feature fusion subnetwork, and a target detection head subnetwork.

[0039] Among them, the image feature extraction subnetwork includes convolution layer, activation layer, batch normalization layer, residual connection layer, and spatial pyramid pooling layer.

[0040] The feature fusion subnetwork includes upsampling layer, skip connection, feature pyramid network layer, and path aggregation network layer.

[0041] The target detection head network includes a convolutional prediction layer, an anchor box matching mechanism, a Sigmoid activation layer, and NMS post-processing.

[0042] The pre-training steps of the YOLOv5 model are as follows: first, a target detection image dataset containing various construction scenes and personnel postures is constructed, and each image is manually annotated, including the bounding box position of the worker in the image and its corresponding category label; then, the annotated data is input into the training process of the YOLOv5 model, and image resizing, data augmentation (such as random cropping, flipping, lighting perturbation, etc.) and batch construction are performed in sequence; during the training phase, the image feature extraction subnetwork of the YOLOv5 model first performs multi-level convolution and downsampling operations on the input image to extract multi-scale semantic feature maps; the feature fusion subnetwork fuses the features of each layer across scales to improve the model's detection ability for targets of different sizes; the target detection head subnetwork performs bounding box regression and category confidence prediction based on the fused feature maps; during the training process, a multi-objective loss function is used as the optimization target, and the bounding box position error, category classification error and confidence error are jointly considered. The model parameters are iteratively optimized through backpropagation and gradient descent algorithms until the training error converges, forming a YOLOv5 pre-training model for personnel target recognition in high-risk areas of construction sites.

[0043] like Figure 2As shown in FIG, based on the pre-trained target detection model, the specific steps of judging whether there are workers in the high-risk area of the construction site are as follows: In the image feature extraction subnetwork of the YOLOv5 model, the regional image data of the high-risk area of the construction site is received, and normalized and resized to generate a standardized image tensor, and multi-level convolution operations and downsampling are performed to extract the spatial texture features and edge gradient features in the construction site image to obtain a multi-scale semantic feature map of the construction image. The specific steps are: In the convolution layer of the image feature extraction subnetwork, the standardized image tensor is locally convolved. The edge response of the region is extracted to obtain the underlying texture features of the image; in the activation layer of the image feature extraction sub-network, the convolution feature map is nonlinearly transformed to enhance the model's ability to express complex boundaries and local patterns; in the batch normalization layer of the image feature extraction sub-network, the mean and variance normalization processing is performed on each convolution output channel to improve the stability and convergence speed of network training; in the residual connection layer of the image feature extraction sub-network, the channel information path between the original features and the deep semantics is retained to avoid the loss of deep information; in the spatial pyramid pooling layer of the image feature extraction sub-network, the feature map is multi-processed. The maximum pooling processing of the scale space window enhances the model's robustness in detecting human targets of different sizes; finally, the multi-scale semantic feature map of the construction image is output for subsequent target fusion analysis; in the feature fusion sub-network of the YOLOv5 model, the multi-scale semantic feature map of the construction image is subjected to cross-layer connection processing and feature channel splicing processing, and the high-resolution structural features and deep semantic information in the image are fused to obtain the fused multi-level feature map. The specific steps are as follows: in the upsampling layer of the feature fusion sub-network, the low-resolution deep semantic features are upsampled to make their size consistent with the high-resolution feature map Matching; In the skip connection structure of the feature fusion sub-network, the upsampled deep semantic features and shallow structural detail features are spliced in the channel dimension to fuse image expression information at different levels; In the feature pyramid network layer of the feature fusion sub-network, multi-layer feature information is propagated from top to bottom to improve the model's responsiveness to large-scale personnel targets; In the path aggregation network layer of the feature fusion sub-network, semantically rich shallow features are aggregated from bottom to top to effectively enhance the detection effect of small-scale personnel targets; Finally, the fused multi-level feature map is output, which contains the structure and position information of workers at different scales;In the target detection sub-network of the YOLOv5 model, the fused multi-level feature map is subjected to convolution decoding, and bounding box regression and category confidence scoring are performed at different detection scales to obtain the detection frame set, confidence value and category identification of each candidate target in the construction site image, and determine whether there are workers in the high-risk area of the construction site. The specific steps are as follows: in the convolution prediction layer of the target detection sub-network, the target category and position are predicted and decoded for the fused feature map of each scale, and the regression and classification results of the candidate targets are generated; in the anchor box matching mechanism of the target detection sub-network, the target category and position are predicted and decoded according to the anchor box matching mechanism. The optimal bounding box candidate region is matched based on the IoU value between the preset anchor box set and the actual feature map prediction box. In the sigmoid activation layer of the target detection head subnetwork, the predicted target confidence and category score are probabilistically normalized to output a standardized target detection result. In the non-maximum suppression (NMS) processing layer of the target detection head subnetwork, all candidate detection boxes are confidence-screened and redundancy is removed, retaining the most representative human target detection box. Finally, the detection box set, confidence value, and category identifier of each candidate target in the construction site image are output to determine whether there are workers in high-risk areas of the construction site.

[0044] In this implementation, the introduction of the YOLOv5 target detection model demonstrates significant advantages in identifying workers in high-risk areas on construction sites. Compared to traditional methods based on optical flow or background subtraction, this model, through the collaborative work of an image feature extraction subnetwork, a feature fusion subnetwork, and a target detection head subnetwork, can extract multi-scale semantic features, fuse deep and shallow structural information, and combine anchor box matching with a non-maximum suppression mechanism to achieve high-precision and robust human target detection. Its structural design adapts to complex environments such as large illumination variations, severe occlusions, and targets of varying sizes at construction sites, and possesses excellent small target detection capabilities and real-time response capabilities. The model constructed through the pre-training process can directly output the bounding box, category label, and confidence value of each candidate target in the input image, thereby accurately determining whether there are workers in high-risk areas. This technology significantly improves recognition accuracy and system stability, providing a solid data foundation for subsequent operational behavior analysis and risk warning, and addressing the problem of traditional methods failing to or misjudging when the target is stationary or subject to environmental interference.

[0045] Specifically, the working image data includes pixel values and two-dimensional coordinates of several pixel points. The feature extraction model is specifically a human posture estimation model, which includes an image feature extraction layer, a posture encoding layer, and a key point prediction head.

[0046] Among them, the image feature extraction layer includes convolution layer, batch normalization layer, activation function layer, residual connection structure, and downsampling module.

[0047] The posture encoding layer includes a multi-scale feature fusion layer, a graph structure convolution, and a heat map generation module.

[0048] The key point prediction head includes convolutional prediction layer, non-maximum suppression, and Sigmoid activation.

[0049] The pre-training steps of the human pose estimation model are as follows: first, a dataset of image keypoint annotations is constructed, including standard action poses, work scene poses, and abnormal poses. The human structure of 17 or 33 key points of the workers in each image is annotated, including the two-dimensional coordinates and visibility labels of key joints such as the head, shoulders, elbows, wrists, hips, knees, and ankles. Then, the pose estimation model is trained using a standard training process, which includes image normalization, data augmentation (such as affine transformation and occlusion simulation), and batch training data construction. During training, the image feature extraction layer performs multi-level convolution on the input image to extract low-level spatial features. The pose encoding layer establishes spatial dependencies between key points through multi-scale feature fusion and graph structure convolution. The heat map generation module generates a response heat map for each key point based on the feature map. In the key point prediction stage, a loss function such as mean squared error or KL divergence is used to measure the difference between the heat map prediction value and the true annotation. The network parameters are optimized through back propagation until the model converges on the validation set, forming a human pose estimation pre-training model for construction site worker posture modeling.

[0050] like Figure 3 As shown, the specific steps for analyzing the work feature set of workers in high-risk areas of the construction site are as follows: the work image data of the workers at several time points are respectively input into the pre-trained human posture estimation model for feature extraction processing to obtain the two-dimensional coordinates and confidence of several key points (such as nose, left eye, right eye, etc.) of the workers at each time point; based on the two-dimensional coordinates and confidence of several key points of the workers at each time point, the work feature set of the workers in high-risk areas of the construction site is analyzed, and the work feature set includes motion amplitude, posture deviation angle, joint movement frequency, vertical displacement change rate, number of motion mutations, posture symmetry index, confidence mean, body center of gravity trajectory change, and joint point jitter index.

[0051] The motion amplitude is obtained by calculating the average Euclidean distance change of all key points of the operator at adjacent time points, and the diagonal length of the image is used as the normalization benchmark;

[0052] The posture deviation angle is obtained by calculating the angle between the key point triples (such as shoulder-elbow-wrist), comparing it with the working posture angle under standard working conditions, and normalizing it with the maximum allowable deviation angle;

[0053] The joint movement frequency is obtained by counting the number of significant changes in the movement amplitude per unit time and dividing it by the sampling time length, and then normalized based on the historical maximum frequency;

[0054] The vertical displacement change rate is calculated by extracting the movement rate of the torso key points (such as chest, waist, and hip) in the Y-axis direction and normalizing it based on the image height;

[0055] The number of motion mutations is counted by detecting whether the motion amplitude exceeds the set fluctuation threshold between consecutive frames;

[0056] The posture symmetry index is constructed by analyzing the relative distance differences between the left and right symmetrical key points of the operator (such as left and right shoulders, left and right knees, etc.) and normalized based on the image width;

[0057] The confidence mean is obtained by taking the arithmetic average of the confidences of all key points, which is itself a unit-normalized probability value;

[0058] The change in the trajectory of the body's center of gravity is extracted by analyzing the curvature or speed trend of the movement trajectory of key core points (such as the hip and waist), and is normalized by the relative trajectory change amplitude;

[0059] The joint jitter index is obtained by calculating the standard deviation of the coordinate fluctuation of each key point in a time series and normalizing it with the average coordinate scale.

[0060] In this implementation, a human posture estimation model is introduced to extract structured features from worker images, achieving higher motion recognition accuracy and behavioral analysis capabilities. Compared with traditional fuzzy judgment methods based on image contours or optical flow, this method can accurately output the coordinates and confidence of key points at each time point, and construct multi-dimensional behavioral parameters such as motion amplitude, posture offset angle, joint motion frequency, and vertical displacement change rate based on continuous time series data to form a fine-grained operation feature set. These parameters are combined to further calculate structured indicators such as posture anomaly index, movement intensity index, and operation instability index, enabling the system to comprehensively assess the risk status of workers from both static posture and dynamic behavior levels. Through normalization processing and fusion of dynamic indicators such as key point jitter and center of gravity trajectory, this method still has good adaptability and robustness in the face of complex situations such as worker stillness, occlusion, and local anomalies, providing a stable and explainable behavioral basis for subsequent risk index modeling and early warning judgment, effectively improving the system's response accuracy and intelligent analysis capabilities to abnormal conditions at the construction site.

[0061] Specifically, the specific steps for obtaining the two-dimensional coordinates and confidence of several key points of the operator at each time point are as follows: in the image feature extraction layer of the human posture estimation model, the work image data of the operator at several time points are received, and the image data are normalized and resized respectively to generate a standardized image tensor, and the standardized image tensor is subjected to multi-level convolution operations, downsampling processing and nonlinear activation processing to extract the underlying texture features, edge response features and spatial structure features of the work image to obtain a multi-scale semantic feature map. The specific steps are: in the convolution layer of the image feature extraction layer, the standardized image tensor is subjected to local image region feature extraction to obtain edge, texture and regional brightness variation. information; in the batch normalization layer of the image feature extraction layer, each channel in the convolution feature map is mean normalized and variance adjusted to improve the numerical stability of the model during training; in the activation function layer of the image feature extraction layer, the normalized features are enhanced by nonlinear mapping, so that the model can effectively characterize complex edges and motion contour changes; in the residual connection structure of the image feature extraction layer, the input features are connected to the deep features across layers to retain the low-level image details and prevent the information from attenuating in the deep network; in the downsampling module of the image feature extraction layer, the feature map is spatially compressed to obtain image semantic features of different scales, and a multi-scale semantic feature map is output for subsequent posture modeling; in In the posture coding layer of the human posture estimation model, the multi-scale semantic feature map is subjected to feature fusion processing and structural relationship modeling, and the spatial connection relationship between the key parts of the operator's body is constructed to obtain a structural expression map containing the semantic information of the operator's whole body skeleton structure. The specific steps are as follows: in the multi-scale feature fusion layer of the posture coding layer, the feature maps of different resolutions are subjected to channel splicing and weighted merging processing to fuse the shallow detail information and deep semantic information in the image; in the graph structure convolution layer of the posture coding layer, a graph relationship model between key points is constructed based on the human skeleton topology structure, and the spatial connection relationship between key points is modeled and feature propagated; in the heat map generation module of the posture coding layer, according to the fusion The key point heat map is constructed by combining the response strength of the features in space, and a probability distribution response map of each key point position is formed, and a structural expression map containing the semantic relationship of the operator's skeleton structure is output; in the key point prediction head of the human posture estimation model, multi-channel convolution prediction processing and probability normalization processing are performed on the structural expression map, and a key point heat map and confidence map are generated in each key point channel respectively. The coordinates of the key point position are analyzed based on the heat map response value, and the two-dimensional coordinates and confidence of several key points of the operator at each time point are output. The specific steps are as follows: in the convolution prediction layer of the key point prediction head, each key point channel is convolved respectively to predict the spatial thermal response of the key point in the image;In the non-maximum suppression (NMS) module of the keypoint prediction head, multiple high-response regions are screened for local maxima, retaining the strongest candidate region corresponding to each keypoint and eliminating redundant prediction points. In the sigmoid activation layer of the keypoint prediction head, the prediction results are probabilistically normalized to generate a heatmap response value and confidence score for each keypoint in the image. Finally, a coordinate analysis operation is performed based on the maximum response position of the keypoint heatmap, outputting the two-dimensional coordinate positions of several keypoints of the operator (including but not limited to the nose, eyes, shoulders, elbows, wrists, hips, knees, ankles, etc.) in the image plane and their corresponding confidence values at each time point.

[0062] In this implementation, a human posture estimation model is used to achieve high-precision extraction of key points of workers, greatly improving the accuracy and robustness of construction site behavior recognition. The model, consisting of an image feature extraction layer, a posture encoding layer, and a key point prediction head, can extract structural information from the image layer by layer, construct skeleton relationships, and output the two-dimensional coordinates and confidence levels of key points. Specifically, the image feature extraction layer uses convolution, activation, normalization, and residual structure to extract underlying texture and edge features. The posture encoding layer constructs the spatial connection semantics between key points through a graph-structured convolution and heat map generation module. The key point prediction head performs convolution prediction and normalization on each channel's heat map and obtains the optimal position coordinates through non-maximum suppression. Ultimately, this method can accurately identify key joints such as the nose, eyes, shoulders, wrists, and knees, and output structured information containing spatial position and confidence levels. It is suitable for construction scenarios such as multi-person work, lighting interference, and complex occlusion. Compared with traditional image difference or motion blur extraction methods, this method has stronger interpretability and higher detection stability, providing key foundational support for subsequent action analysis and risk assessment.

[0063] Specifically, the specific steps for analyzing the work characteristic index set of workers in high-risk areas of the construction site are as follows: based on the motion amplitude, joint movement frequency, and confidence mean of the workers in high-risk areas of the construction site, analyze the motion intensity index of the workers in high-risk areas of the construction site; based on the posture deviation angle, number of motion mutations, posture symmetry index, and confidence mean of the workers in high-risk areas of the construction site, analyze the posture abnormality index of the workers in high-risk areas of the construction site; based on the vertical displacement change rate, body center of gravity trajectory change, and joint point jitter index of the workers in high-risk areas of the construction site, analyze the operation instability index of the workers in high-risk areas of the construction site.

[0064] The specific formula for calculating the intensity index of workers in high-risk areas of the construction site is as follows: DzJ = ln(1+DzF*DzP)*(1+e -ZxJ); wherein DzJ is the intensity index of the movements of the workers in the high-risk areas of the construction site, DzF is the amplitude of the movements of the workers in the high-risk areas of the construction site, DzP is the frequency of the joint movements of the workers in the high-risk areas of the construction site, e is a natural constant, which is 2.71 in this embodiment, and ZxJ is the mean confidence level of the workers in the high-risk areas of the construction site.

[0065] The specific formula for calculating the abnormal posture index of workers in high-risk areas of construction sites is as follows: Among them, ZtY is the posture abnormality index of the workers in the high-risk area of the construction site, ZtP is the posture deviation angle of the workers in the high-risk area of the construction site, DzT is the posture symmetry index of the workers in the high-risk area of the construction site, DzT is the number of sudden changes in the movements of the workers in the high-risk area of the construction site, ZxJ is the mean confidence value of the workers in the high-risk area of the construction site, and δ is the confidence adjustment factor stored in the database, which is 0.1 in this embodiment.

[0066] The specific formula for calculating the operational instability index of workers in high-risk areas of construction sites is as follows: Among them, CzS is the operational instability index of the workers in the high-risk area of the construction site, e is a natural constant, which is 2.71 in this embodiment, CzW is the vertical displacement change rate of the workers in the high-risk area of the construction site, DdZ is the joint point jitter index of the workers in the high-risk area of the construction site, μ is the jitter influence coefficient stored in the database, and ZxG is the change in the body center of gravity trajectory of the workers in the high-risk area of the construction site.

[0067] It should be explained that the specific steps for obtaining the jitter influence coefficient μ stored in the database are: first, continuous image data of construction workers under different working conditions are collected, and the two-dimensional coordinate sequence of key joints (such as wrists, knees, ankles, etc.) between continuous time points is extracted based on the human posture estimation model; secondly, the standard deviation of the coordinate change of each key point per unit time is calculated as the jitter amplitude indicator of the key point; then, combined with historical accident case data, the correspondence between the jitter amplitude of different key points and the operation abnormality or risk level is statistically analyzed, and an influence mapping model between the jitter amplitude of key points and operation stability is constructed; further, regression analysis or weight evaluation method is used to extract the average impact of the overall jitter on operation stability, and the value is stored as the jitter influence coefficient μ for subsequent modeling and calculation of the operation instability index.

[0068] In this implementation, a set of operation characteristic indices is constructed. Based on multi-dimensional behavioral parameters such as motion amplitude, posture deviation angle, and vertical displacement change rate, the motion intensity index, posture abnormality index, and operational instability index are calculated respectively to achieve a quantitative assessment of the operator's behavioral status. Each index constructs an innovative formula by integrating the nonlinear relationship of key parameters, taking into account variables such as confidence and jitter influence coefficient, thereby enhancing the model's sensitivity and robustness to complex behavioral patterns. Among them, the motion intensity index combines motion amplitude and frequency to characterize the risks of high-intensity operations; the posture abnormality index integrates deviation angle, symmetry, and number of mutations to accurately identify situations where postures violate regulations; and the operational instability index determines the trend of falling or sliding through the center of gravity trajectory and joint jitter analysis. More importantly, the relevant influence coefficients are based on actual working condition data and accident label statistical modeling to ensure that the model weight distribution has scenario adaptability and engineering reliability. This method is not only highly interpretable, but also can classify and trace the sources of risk, significantly improving the intelligent and structured level of construction behavior monitoring and providing a solid numerical foundation for the early warning system.

[0069] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0070] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. Construction operation monitoring and early warning system based on image processing, characterized by: include: The recognition and analysis unit is used to obtain regional image data of high-risk areas of the construction site in real time and determine whether there are workers in the high-risk areas of the construction site based on the pre-trained target detection model; An operation image acquisition unit is used to continuously acquire operation image data of the operators at several time points when the operators are present in the high-risk area of the construction site; An operation image analysis unit is used to combine the operation image data of the operators at several time points with a pre-trained feature extraction model to analyze the operation feature set of the operators in the high-risk area of the construction site, and analyze the operation feature index set of the operators in the high-risk area of the construction site, wherein the operation feature index set includes the movement intensity index, the posture abnormality index, and the operation instability index; An operation risk analysis unit is used to analyze the comprehensive operation risk index of operators in high-risk areas of the construction site based on the operation characteristic index set of operators in high-risk areas of the construction site, and make judgments and analyses with the preset operation risk range; The operation risk warning unit is used to send an operation risk alert when the comprehensive operation risk index of operators in high-risk areas of the construction site is within a preset operation risk range.

2. The construction operation monitoring and early warning system based on image processing according to claim 1 is characterized in that: The regional image data includes pixel values of a plurality of regional pixel points. The target detection model is specifically a YOLOv5 model. The YOLOv5 model includes an image feature extraction subnetwork, a feature fusion subnetwork, and a target detection head subnetwork.

3. The construction operation monitoring and early warning system based on image processing according to claim 2 is characterized in that: Based on the pre-trained target detection model, the specific steps for determining whether there are workers in high-risk areas of the construction site are as follows: The image feature extraction subnetwork of the YOLOv5 model receives regional image data from high-risk areas of the construction site, performs normalization and resizing, generates a standardized image tensor, and then performs multi-level convolution and downsampling operations to extract spatial texture features and edge gradient features from the construction site image, resulting in a multi-scale semantic feature map of the construction image. In the feature fusion subnetwork of the YOLOv5 model, cross-layer connection processing and feature channel splicing are performed on the multi-scale semantic feature maps of the construction images, fusing the high-resolution structural features and deep semantic information in the images to obtain a fused multi-level feature map. In the target detection head subnetwork of the YOLOv5 model, convolutional decoding is performed on the fused multi-level feature maps, and bounding box regression and category confidence scoring are performed at different detection scales. The detection box set, confidence value, and category identification of each candidate target in the construction site image are obtained, and the presence of workers in high-risk areas of the construction site is determined.

4. The construction operation monitoring and early warning system based on image processing according to claim 1 is characterized in that: The operation image data includes pixel values and two-dimensional coordinates of several pixel points. The feature extraction model is specifically a human posture estimation model, which includes an image feature extraction layer, a posture encoding layer, and a key point prediction head.

5. The construction operation monitoring and early warning system based on image processing according to claim 4 is characterized in that: The specific steps for analyzing the operation feature set of workers in high-risk areas of the construction site are as follows: The work image data of the workers at several time points are input into the pre-trained human posture estimation model for feature extraction, and the two-dimensional coordinates and confidence levels of several key points of the workers at each time point are obtained; Based on the two-dimensional coordinates and confidence levels of several key points of the workers at each time point, the work feature set of the workers in high-risk areas of the construction site is analyzed. The work feature set includes movement amplitude, posture deviation angle, joint movement frequency, vertical displacement change rate, number of movement mutations, posture symmetry index, confidence mean, change in body center of gravity trajectory, and joint point jitter index.

6. The construction operation monitoring and early warning system based on image processing according to claim 5 is characterized in that: The specific steps to obtain the two-dimensional coordinates and confidence levels of several key points of the operator at each time point are as follows: In the image feature extraction layer of the human posture estimation model, work image data of workers at several time points are received, and the image data are normalized and resized to generate a standardized image tensor. The standardized image tensor is subjected to multi-level convolution operations, downsampling processing, and nonlinear activation processing to extract the underlying texture features, edge response features, and spatial structure features of the work image to obtain a multi-scale semantic feature map; In the posture encoding layer of the human posture estimation model, feature fusion processing and structural relationship modeling are performed on the multi-scale semantic feature map, and the spatial connection relationship between the key parts of the operator's body is constructed to obtain a structural expression map containing the semantic information of the operator's full-body skeletal structure; In the key point prediction head of the human posture estimation model, the structural expression graph is subjected to multi-channel convolution prediction processing and probability normalization processing, and a key point heat map and confidence map are generated in each key point channel respectively. The key point position coordinates are analyzed based on the heat map response value, and the two-dimensional coordinates and confidence of several key points of the operator at each time point are output.

7. The construction operation monitoring and early warning system based on image processing according to claim 5 is characterized in that: The specific formula for calculating the comprehensive operational risk index for workers in high-risk areas of a construction site is as follows: Among them, ZyF, DzJ, ZtY, CzS, DzF, and ZtP are the comprehensive operation risk index, movement intensity index, posture abnormality index, operation instability index, movement amplitude, and posture deviation angle of the workers in the high-risk area of the construction site, respectively. λ1, λ2, and λ3 are the movement intensity influence coefficient, posture abnormality influence coefficient, and operation instability influence coefficient stored in the database, respectively.

8. The construction operation monitoring and early warning system based on image processing according to claim 5 is characterized in that: The specific steps for analyzing the work characteristic index set of workers in high-risk areas of the construction site are as follows: Based on the movement amplitude, joint movement frequency, and confidence mean of workers in high-risk areas of the construction site, the movement intensity index of workers in high-risk areas of the construction site is analyzed; Based on the posture deviation angle, number of sudden changes, posture symmetry index, and confidence mean of workers in high-risk areas of the construction site, the posture abnormality index of workers in high-risk areas of the construction site is analyzed; The operational instability index of workers in high-risk areas of the construction site is analyzed based on the vertical displacement change rate, body center of gravity trajectory change, and joint point jitter index of workers in high-risk areas of the construction site.

9. The construction operation monitoring and early warning system based on image processing according to claim 8 is characterized in that: The specific formula for calculating the motion intensity index of workers in high-risk areas of construction sites is as follows: DzJ=ln(1+DzF*DzP)*(1+e -ZxJ ); Among them, DzJ, DzF, DzP, and ZxJ are the motion intensity index, motion amplitude, joint movement frequency, and confidence mean of workers in high-risk areas of the construction site, respectively, and e is a natural constant.

10. The construction operation monitoring and early warning system based on image processing according to claim 8, characterized in that: The specific formula for calculating the abnormal posture index of workers in high-risk areas of construction sites is as follows: Among them, ZtY, ZtP, DzT, DzT, and ZxJ are the posture abnormality index, posture deviation angle, posture symmetry index, number of action mutations, and confidence mean of the workers in the high-risk area of the construction site, respectively, and δ is the confidence adjustment factor stored in the database.

Images