Method and system for automatic identification and evaluation of construction risks of high-altitude scaffold operation personnel

Abstract

This invention discloses a method and system for automatic identification and assessment of construction risks for workers operating on high-altitude scaffolding, belonging to the field of construction risk identification technology. The method includes the following steps: multi-category target detection, using a target detection model to detect and locate workers, personal protective equipment, and scaffolding structures; generating scaffolding segmentation masks using an image segmentation network; detecting key points of the human skeleton using a human pose estimation model; high-risk action classification, designing a dual-path action classification strategy, including deep learning-based skeleton image classification and pose parameter classification based on artificial features, combining key points of the human skeleton to determine the action category, and checking whether the key points of the human skeleton overlap with the scaffolding area to correct the action classification results; risk index calculation and comprehensive assessment. This invention achieves automatic detection, identification, and risk quantification assessment of unsafe behaviors of personnel in complex construction sites, providing intelligent auxiliary means for construction safety supervision.

Description

Technical Field

[0001] This invention belongs to the field of construction risk identification technology, and in particular relates to a method and system for automatic identification and assessment of construction risks for workers operating at heights on scaffolding. Background Technology

[0002] High-altitude construction work is a hazardous industry, and the safety of construction workers is a key concern. Instability of the work platform (e.g., improper scaffolding erection, damaged or loosely secured suspended platforms), lack of or malfunctioning protective equipment, operator error, and environmental factors can all lead to construction risks. The core risks of high-altitude construction work are falls, falling objects, environmental impacts, and equipment failure. Among these, operator risks, especially those involving high-altitude scaffolding workers, require focused control. Currently, to reduce fall rates, regulations mandate that workers use or wear fall protection equipment. However, this does not completely eliminate risks from operator error and external work platforms, particularly in complex construction sites where the risks are even greater. Therefore, there is an urgent need for a technology that can automatically detect, identify, and quantify the risks of unsafe acts at complex construction sites, providing intelligent support for construction safety supervision.

[0003] This invention proposes a method and system for automatic identification and assessment of construction risks for workers operating at heights. It designs an "automatic identification and assessment framework for construction risks of workers operating at heights" that integrates multiple visual models. The framework links target detection, semantic segmentation, posture estimation, action classification and multi-index decision-making. By detecting construction workers and PPE (Personal Protective Equipment), segmenting the scaffold structure, estimating the posture of the personnel skeleton and identifying abnormal actions, the risk level of each worker is comprehensively assessed and ranked. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for automatic identification and assessment of construction risks for workers operating at heights and scaffolding, in order to solve the problems mentioned in the background art, such as the lack of automatic detection, identification and risk quantification assessment of unsafe behaviors of personnel in complex construction sites.

[0005] To achieve the above objectives, the present invention employs the following technical solution:

[0006] In its first aspect, this invention proposes a method for automatic identification and assessment of construction risks for workers operating scaffolding at heights, comprising the following steps:

[0007] S1. Multi-category target detection: Construct a target detection model to detect and locate workers, personal protective equipment, and scaffolding structures from the original monitoring images;

[0008] S2. Scaffolding region segmentation: Based on the location markers of the scaffolding structure in the monitoring image, an image segmentation network is used to generate a scaffolding segmentation mask to obtain the scaffolding region segmentation result;

[0009] S3. Human pose estimation: Based on the positioning marks of workers in the monitoring images, the key points of the human skeleton are detected by the human pose estimation model.

[0010] S4. High-risk action classification: Design a dual-path action classification strategy, including deep learning-based skeleton image classification and pose parameter classification based on artificial features. Combine the key points of the human skeleton to determine the action category. Compare and analyze the dual-path action classification results to evaluate the accuracy of the action classification results. In addition, design a structure-assisted discrimination fusion strategy, using the scaffolding region segmentation results to provide spatial reference, check whether there is pixel overlap between the key points of the human skeleton and the scaffolding region, and correct the action classification results.

[0011] S5. Risk Indicator Calculation and Comprehensive Assessment: Select the degree of action risk, the degree of personal protective equipment wearing, the working height, and the stability of the scaffold structure as core risk indicators to construct a risk indicator system. Normalize the core risk indicators, calculate ANP weights, and calculate the comprehensive vector to obtain the ELECTRE III comprehensive ranking.

[0012] Preferably, the target detection model in S1 is as follows:

[0013] The target detection model is built on the YOLOv8 architecture;

[0014] Given an input image The object detection model first extracts multi-scale feature maps through a feature extraction network; then, a feature fusion module aggregates the multi-scale features, and the detection head outputs a set of... Prediction results:

[0015]

[0016] in, This indicates the center coordinates and dimensions of the predicted bounding box. Let be the class probability vector of the predicted bounding box. The probability of the existence of the target;

[0017] The set of detection categories for the object detection model is as follows:

[0018]

[0019] in, This indicates the test results from the operators. This indicates the test results for personal protective equipment. This indicates the inspection results of the scaffolding structure.

[0020] Preferably, the image segmentation network in S2 is as follows:

[0021] The image segmentation network is built based on the SAM2 framework;

[0022] During training, the image backbone encoder of SAM2 is frozen, and only the parameters of the mask decoder and cue embedding layer are fine-tuned; and the optimization objective during training consists of a weighted sum of binary cross-entropy loss and mask IoU loss.

[0023] A multi-point cue fusion strategy is introduced into the trained and optimized image segmentation network for segmentation:

[0024] The candidate bounding boxes for scaffolding structures output by the object detection model As the initial cue box for the image segmentation network, cue samples are randomly sampled inside and outside the box to form a set. For the same input image, generate Group random point prompt The masks were predicted respectively. Final output:

[0025]

[0026] in, This is the final probability mask after fusion. This is a pixel-level probability map of the predicted scaffolding.

[0027] Preferably, the human pose estimation model in S3 is as follows:

[0028] The human pose estimation model is constructed by adding a keypoint prediction branch to the traditional YOLO detection framework.

[0029] Preferably, the deep learning-based skeleton image classification in S4 is as follows:

[0030] ResNet-18 is used as the base model for skeleton image classification. The coordinates of 17 key points output by the human pose estimation model are visualized, each key point is marked and connected to form a skeleton image. The skeleton image is then input into the ResNet-18 network for classification, including five types of actions: standing, sitting, squatting, climbing, and leaning. The action classification results are then output.

[0031] Preferably, the pose parameter classification based on artificial features in S4 is as follows:

[0032] The posture feature vector is calculated using human joint coordinates. The joint angle features, trunk principal axis direction angle, shoulder width, height-to-width ratio, center of gravity position, and symmetry score are calculated respectively. The joint angle features include the angle between the left and right upper limbs formed by elbow-shoulder-wrist, the angle between the left and right lower limbs formed by hip-knee-ankle, the angle between the left and right shoulders formed by elbow-shoulder-hip, and the left and right trunk bending angle formed by shoulder-hip-knee. All features are normalized and combined to form the posture feature vector.

[0033] The pose feature vectors are classified using traditional machine learning models, which may include one or more of the following: SVM model, Random Forest (RF) and XGB classifier.

[0034] Furthermore, the joint angle features are as follows:

[0035] For the three key points ,by As the vertex, the included angle Defined as:

[0036]

[0037] in, The key point coordinate vector, Represents the vector dot product. This represents the magnitude of the vector.

[0038] Furthermore, the orientation angle of the main axis of the torso:

[0039] Using the midpoint of the shoulder Midpoint of the hip Constitutes the principal axis vector of the torso , and the vertical unit vector The included angle is:

[0040]

[0041] This feature is used to determine whether the body is tilted or lying down.

[0042] Furthermore, the shoulder width:

[0043] left shoulder with right shoulder Euclidean distance:

[0044]

[0045] This feature reflects the degree to which the upper body is open.

[0046] Furthermore, the aspect ratio:

[0047] The height of the human body detection frame is Width is The aspect ratio is This feature is used to distinguish between standing and crouching postures.

[0048] Furthermore, the location of the center of gravity:

[0049] Key point set The average coordinates are:

[0050]

[0051] ordinate with the center of the detection frame offset It is used to determine whether the body's center of gravity has shifted downwards.

[0052] Furthermore, the symmetry score:

[0053] The average Euclidean distance between the left and right pairs of keypoints:

[0054]

[0055] in, For symmetrical keypoints (such as shoulders, elbows, knees, etc.), this feature is used to characterize the degree of left-right balance in body posture.

[0056] Preferably, in step S4, checking whether there is pixel overlap between the key points of the human skeleton and the scaffolding area is as follows:

[0057] Let the set of key points output by the human pose estimation model be:

[0058]

[0059] in, For the first Pixel coordinates of key points;

[0060] Overlap determination is expressed by the following rules:

[0061]

[0062] in, For index functions, The final mask for the scaffolding area. For the coordinates of the key points, The threshold for pixel overlap; if it satisfies Then it is determined that there is an interaction relationship between the key point and the scaffolding area;

[0063] If the key point overlaps with the scaffolding area, the action category is valid; otherwise, it is a potential misjudgment.

[0064] Preferably, the risk indicator system in S5 is as follows:

[0065] The risk level of an action is mapped from the action category output by the high-risk action classification to the corresponding risk score.

[0066] The degree of personal protective equipment (PPE) wearing is determined by the detection results of PPE in the target detection model. PPE includes safety helmets, vests, and seat belts, and the number of items not worn is mapped to the corresponding risk score.

[0067] The working height is determined by the relative vertical position of the worker detection box in the target detection model within the scaffold detection box, and is then mapped to the corresponding risk score.

[0068] The stability of the scaffold structure is mapped to the corresponding risk score by the boundary overlap rate between the scaffold segmentation mask and the standard profile.

[0069] In a second aspect, this invention proposes an automatic risk identification and assessment system for workers operating high-altitude scaffolding, comprising:

[0070] The multi-category target detection module uses the YOLOv8 model to detect and locate workers, personal protective equipment, and scaffolding structures in the input raw monitoring images, and outputs predicted bounding boxes for workers, personal protective equipment, and scaffolding structures.

[0071] The scaffolding region segmentation module is designed based on the SAM2 framework and a multi-point cue fusion strategy. It is used to generate a scaffolding segmentation mask by using the predicted bounding boxes of the scaffolding structure output by the multi-class object detection module as the initial cue boxes.

[0072] The pose estimation module uses the YOLO-Pose model, which is built by adding a key point prediction branch to the traditional YOLO detection framework. The predicted bounding boxes of the workers output by the multi-class target detection module are clipped and input into the YOLO-Pose model. The YOLO-Pose model outputs 17 key points of the human skeleton.

[0073] The high-risk action classification module includes a skeleton image classification model based on ResNet-18 and a pose parameter classification model based on SVM, Random Forest (RF), and XGB classifiers, respectively. Both the skeleton image classification model and the pose parameter classification model combine the key points of the human skeleton to determine the action category. The action category output by the model with the highest accuracy is taken as the action classification result. Furthermore, the action classification result is corrected by checking whether there is pixel overlap between the key points of the human skeleton and the scaffolding segmentation mask.

[0074] The risk indicator calculation and comprehensive assessment module establishes a risk assessment system, combining ANP weight calculation with ELECTREIII ranking to quantitatively rank the risk status of multiple workers.

[0075] Compared with the prior art, the beneficial effects of the present invention are:

[0076] (1) The model framework of the method in this invention uses the latest YOLOv8 model to detect personnel and safety equipment, and combines the YOLO-Pose extended network to complete the pose estimation of 17 key points of the human body; it integrates SAM2 (Segment Anything Model v2) to perform fine semantic segmentation of the scaffolding area; it extracts the skeleton geometric features to train SVM (Support Vector Machine), Random Forest (RF) and XGB (XGBoost) classifiers to identify high-risk actions; and finally, it integrates multiple risk indicators to complete the risk ranking by using the Analytic Network Process (ANP) and ELECTRE III multi-criteria decision model.

[0077] (2) The method proposed in this invention is a multi-module fusion method covering target detection, structure segmentation, posture recognition and risk assessment, which realizes end-to-end identification of risks in high-altitude scaffolding operations; each module is closely connected and assists each other, reducing information redundancy and improving overall accuracy.

[0078] (3) The method in this invention constructs its own image dataset for high-altitude construction scenarios and uses the self-built construction scenario dataset for training and validation. Efficient model training processes are designed for personnel, PPE, scaffold detection, scaffold segmentation, pose estimation and action classification, including data annotation, enhancement strategies, network structure selection and hyperparameter optimization, to verify the performance of the proposed model in complex field environments.

[0079] (4) The method of this invention introduces scaffold structure segmentation as an aid to action discrimination, which realizes accurate identification of actions that rely on support such as "leaning" and "climbing", making up for the shortcomings of judging solely by the human skeleton.

[0080] (5) The method in this invention establishes a risk assessment system that includes four indicators: action risk, PPE status, working height, and structural quality. It combines ANP weight calculation with ELECTRE III ranking to quantitatively rank the risk levels of multiple personnel. The reliability and consistency of the comprehensive evaluation results are verified by comparison with expert manual assessment. Attached Figure Description

[0081] Figure 1This is a flowchart of the automatic identification and assessment method for construction risks of workers operating high-altitude scaffolding in this invention;

[0082] Figure 2 This is a flowchart of the scaffolding area segmentation in this invention;

[0083] Figure 3 This is a flowchart illustrating the classification of high-risk actions in this invention;

[0084] Figure 4 This is a schematic diagram of the angles of the limbs, shoulders, and torso in the traditional classification of posture features in Embodiment 1 of the present invention;

[0085] Figure 5 This is a schematic diagram illustrating the process of creating the sam2 scaffolding segmentation dataset in Embodiment 2 of the present invention;

[0086] Figure 6 This is a flowchart of sam2 segmentation in Embodiment 2 of the present invention;

[0087] Figure 7 This is a flowchart of bone extraction in Embodiment 2 of the present invention;

[0088] Figure 8 This is a skeleton example diagram applicable to ResNet in Embodiment 2 of the present invention;

[0089] Figure 9 This is a schematic diagram of the detection results in scenario 1 of embodiment 2 of the present invention;

[0090] Figure 10 This is a schematic diagram of the detection results in scenario 2 of embodiment 2 of the present invention. Detailed Implementation

[0091] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0092] Example 1:

[0093] The method for automatic identification and assessment of construction risks for high-altitude scaffolding workers mainly involves constructing an automatic identification and assessment model for construction risks for high-altitude scaffolding workers. The model framework includes a multi-category target detection module, a scaffolding area segmentation module, a posture estimation module, a high-risk work identification module, and a comprehensive assessment module. After completing detection, identification, and feature extraction, the comprehensive assessment and ranking are achieved using an analytical network process and the ELECTRE III multi-criteria ranking method.

[0094] See Figure 1 The method includes the following steps:

[0095] Step 1: YOLOv8 object detection.

[0096] A multi-class object detection module is used to locate construction workers, personal protective equipment (PPE), and scaffolding structures from raw surveillance images. This module employs the YOLOv8 architecture, a single-stage object detection model that integrates bounding box regression and object classification in a single forward propagation, achieving real-time detection performance while maintaining high accuracy.

[0097] Specifically, given an input image YOLOv8 first uses a feature extraction network Multi-level feature maps are extracted; subsequently, the feature fusion module aggregates multi-scale features to enhance the detection capability of targets of different sizes. The detection head outputs a set of... Prediction results:

[0098]

[0099] in, This indicates the center coordinates and dimensions of the predicted bounding box. Let be the class probability vector of the predicted bounding box. The objectness score is the probability of the object existing.

[0100] The object detection module uses precision, recall, and mAP (mean average precision) as evaluation metrics. Precision measures the percentage of true positive samples in the detection results and is defined as:

[0101]

[0102] Recall measures how many true positive samples are detected by the model, and is defined as:

[0103]

[0104] in, , , These represent true positive, false positive, and false negative, respectively.

[0105] Based on this, the average precision (AP) is obtained by calculating the area under the precision-recall (PR) curve, and its mathematical form is:

[0106]

[0107] in, Indicates that the recall rate is The precision function is used to approximate the precision rate over time. Since the precision-recall curve is discrete in actual calculations, the precision-recall function is usually approximated by taking a weighted average of the precision rates at different recall rates.

[0108] mAP is defined as the average AP over all categories:

[0109]

[0110] in, Indicates the total number of categories. Indicates category The average precision (mAP) is the most representative comprehensive evaluation metric in the field of object detection, capable of comprehensively reflecting the overall performance of a model in multi-class detection tasks.

[0111] The detection category set for this module is:

[0112]

[0113] The module categorizes targets into three PPE-related categories: helmet, vest, and safety belt; the last category is environmental structural targets. The module's output has two functions:

[0114] (1) The worker bounding box will be clipped and input into the subsequent human pose estimation module;

[0115] (2) PPE test results will be directly converted into protection status indicators for risk assessment. .

[0116] Step 2: SAM2 scaffolding region semantic segmentation.

[0117] To obtain spatial information about the supporting structure where workers are located during risk assessment, this invention designs a scaffolding region segmentation module based on the output of the detection module. This module is based on the Segment AnythingModel v2 (SAM2) framework proposed by Meta. SAM2 is a cue-driven image segmentation network that can generate corresponding target masks based on input cue information (such as points, boxes, etc.). Figure 2 As shown.

[0118] Specifically, let the input image be:

[0119]

[0120] in, and These represent the height and width of the image, respectively. The candidate bounding boxes for the scaffolding output by the detection module are denoted as:

[0121]

[0122] in, and These are the coordinates of the top-left and bottom-right corners of the bounding box. This box serves as the initial cue box for SAM2. Furthermore, random sampling is performed inside and outside this box. Each point is a clue, forming a set:

[0123]

[0124] in, For the first The pixel coordinates of the prompt point The label for the dot tooltip (1 indicates it is in the target area, 0 indicates it is in the background).

[0125] The SAM2 segmentation prediction function can be expressed as:

[0126]

[0127] in, Represents a segmented network. These are model parameters; This is a pixel-level probability map of the predicted scaffolding. Indicates position The probability that a pixel belongs to scaffolding.

[0128] To accommodate the unique structure of slender scaffold members and cross bracing, this invention investigates and freezes the SAM2 image backbone encoder, focusing only on the mask decoder. and prompt embedding layer Fine-tune the parameters:

[0129]

[0130] in, This indicates the frozen feature extraction network parameters. and These are the trainable mask decoder and the cue embedding layer parameters, respectively.

[0131] The optimization objective of the segmentation module consists of a weighted sum of the binary cross-entropy (BCE) loss and the mask IoU loss:

[0132]

[0133] in, This is the actual scaffolding mask (1 indicates the scaffolding area, 0 indicates the background). These are the weighting coefficients for BCE loss and IoU loss, respectively.

[0134] The binary cross-entropy loss is:

[0135]

[0136] in, Indicates position The actual label value; Indicates position The predicted probability.

[0137] The mask IoU loss is:

[0138]

[0139] The numerator is the sum of the number of pixels at the intersection of the predicted and the true masks, and the denominator is the sum of the number of pixels at the union of the two.

[0140] During the inference phase, a multi-point cue fusion strategy is introduced: for the same input image, generate... Group random point prompt The masks were predicted respectively. Final output:

[0141]

[0142] in, This is the final probability mask after fusion.

[0143] Within the overall framework, the resulting segments Post-processing for action classification: Let the set of keypoints output by the human pose estimation module be:

[0144]

[0145] in, For the first The pixel coordinates of each key point. If the following conditions are met:

[0146]

[0147] Then it is determined that there is an interaction relationship between the key point and the scaffolding. (Using a threshold) to help identify high-risk actions such as "leaning" or "climbing". The segmented scaffold area not only serves as a spatial constraint for action classification, but also provides a basis for determining structural integrity and working height during the risk assessment phase.

[0148] Step 3: YOLO-Pose Human Pose Estimation.

[0149] After detecting a person, this invention crops the person area from the original image and inputs it into the pose estimation module to obtain the coordinates of the human body's key points. For this purpose, this invention employs the YOLO-Pose model, which adds a key point prediction branch to the traditional YOLO detection framework, achieving the integration of object detection and pose regression. Unlike the traditional two-stage method (detecting first and then performing pose estimation separately), YOLO-Pose outputs the person's bounding box and 17 key points of the entire body simultaneously through a single forward computation. These 17 key points are shown in Table 1.

[0150] Table 1 Key Skeletal Points of YOLOv8-Pose

[0151]

[0152] Specifically, let the input image be... The bounding boxes of the people obtained through YOLO detection are as follows:

[0153]

[0154] The cropped personnel sub-image is then input into the YOLO-Pose model, and the output keypoint set is...

[0155]

[0156] in, For the first Predicted coordinates of key points.

[0157] This module uses the OKS metric for evaluation. OKS is a similarity measure corresponding to IoU in the keypoint task. Let the set of keypoints of a certain target (person) in an image be denoted as OKS. , No. The predicted coordinates of the key points are: The actual coordinates are Visibility indicator is (1 indicates participation in the evaluation). The OKS for a single objective is then defined as:

[0158]

[0159] in, To detect the Euclidean distance between keypoints and real keypoints; For visibility indicators of key points; The scale of the target (usually the square root of the area of ​​the target's truth box or an equivalent scale). This is the scaling factor. The closer OKS is to 1, the closer the predicted keypoints are to the true values.

[0160] This invention fixes the weights of the YOLO backbone network during training, training only the keypoint branches. Data augmentation includes random affine transformations and flipping to improve the model's robustness. After training, the output keypoint coordinates and confidence scores are saved in JSON format for use by the downstream action classification module.

[0161] Step 4: Classification of high-risk actions.

[0162] After acquiring key points of the human skeleton, this invention designs a dual-path motion classification strategy to determine whether a worker is in a high-risk motion state. The specific strategy includes two methods: one is skeleton image classification based on deep learning, and the other is pose parameter classification based on manually generated features. These two methods complement each other, and comparative analysis is used to evaluate the effectiveness of different methods for various types of motions, such as... Figure 3 As shown.

[0163] 1) ResNet-based skeleton image classification;

[0164] This invention uses ResNet-18 as the base model for skeleton image classification. ResNet (Residual Networks) is a deep convolutional neural network that overcomes the gradient vanishing and degradation problems that occur during the training of deep neural networks by introducing residual connections. In each residual block, the input... and output There is a skip connection between them:

[0165]

[0166] in, It is a series of operations involving convolution, batch normalization, and activation functions. It is input. It is the output. With this structure, the network can directly learn the residual (the difference between the input and the output), rather than directly learning the mapping between the input and the output, which greatly improves the training effect of deep networks.

[0167] In this invention, the coordinates of 17 key points output by the human pose estimation module are visualized, each joint is marked and connected to form a red skeleton image. This skeleton image is input into a ResNet-18 network for classification, outputting 5 types of actions (standing, sitting, squatting, climbing, and leaning). ResNet, through its residual block structure, can effectively extract local features from the skeleton image and capture the spatial structure of the skeleton.

[0168] Assume the output class probability is:

[0169]

[0170] in, This represents the predicted probabilities of five action categories. The loss function used is cross-entropy loss:

[0171]

[0172] in, For real labels, The model predicts the first The probability of a class.

[0173] 2) Traditional classification of posture features

[0174] To provide an alternative classification approach, this invention also designs a traditional classifier based on pose features. This invention calculates pose feature vectors using human joint coordinates and performs classification using machine learning models such as Support Vector Machine, Random Forest, and XGBoost. Let the keypoint coordinates of each person be... ,extract 3D geometric eigenvectors ,include:

[0175] (1) Joint angle characteristics:

[0176] For the three key points ,by As the vertex, the included angle Defined as:

[0177]

[0178] in, The key point coordinate vector, Represents the vector dot product. This represents the vector magnitude. This feature primarily includes the angles between the left and right upper limbs (elbow-shoulder-wrist) and the left and right lower limbs (hip-knee-ankle), as well as the shoulder angle (elbow-shoulder-hip) and the trunk bending angle (shoulder-hip-knee), such as... Figure 4 As shown.

[0179] (2) Main axis direction angle of the torso:

[0180] Using the midpoint of the shoulder Midpoint of the hip Constitutes the principal axis vector of the torso , and the vertical unit vector The included angle is:

[0181]

[0182] This feature is used to determine whether the body is tilted or lying down.

[0183] (3) Shoulder width:

[0184] left shoulder with right shoulder Euclidean distance:

[0185]

[0186] This feature reflects the degree to which the upper body is open.

[0187] (4) Aspect Ratio:

[0188] The height of the human body detection frame is Width is The aspect ratio is .

[0189] This feature is used to distinguish between standing and crouching postures.

[0190] (5) Center of gravity position:

[0191] Key point set The average coordinates are:

[0192]

[0193] ordinate with the center of the detection frame offset Used to determine if the body's center of gravity has shifted downwards (as in a squat).

[0194] (6) Symmetry score:

[0195] The average Euclidean distance between the left and right pairs of keypoints:

[0196]

[0197] in This is the logarithm of symmetrical key points (such as shoulders, elbows, and knees). This feature is used to characterize the left-right balance of body posture.

[0198] All features are normalized to form a feature vector. As input to a traditional classifier, SVM uses the RBF kernel and grid search parameters. RF and XGB were optimized by adjusting the number and depth of trees. This feature path can directly determine the action category without the need for skeleton visualization.

[0199] 3) Structure-assisted discrimination fusion

[0200] To address the limitations of both traditional classification and deep learning classification methods, this invention designs a structure-assisted discriminative fusion strategy. Specifically, considering the unique characteristics of actions such as "leaning" and "climbing" (which rely on interaction with the environment), this invention utilizes a scaffold segmentation mask to provide spatial reference. When the ResNet model determines an action as "climbing" or "leaning," this invention further checks whether key points of the human skeleton (especially the shoulder and back areas) overlap with the scaffold region. If the upper body key points coincide with the mask region, the action determination is considered valid; otherwise, if the skeleton is completely suspended, it is a potential misclassification and should be downgraded. This overlap determination can be represented by the following rules:

[0201]

[0202] in, For index functions, The final mask for the scaffolding area. For the coordinates of the key points, The threshold for pixel overlap (set to 15).

[0203] To ensure that different types of models (ResNet, SVM, RF, XGBoost) are compared under a unified standard, this module uses accuracy as the primary evaluation metric. Assume the dataset contains a total of... There are samples, and the total number of categories is [number]. The accuracy rate is calculated as follows:

[0204]

[0205] in, Indicates category The correct number of predictions; The numerator represents the total number of samples; the denominator represents the total number of samples in which all categories were correctly identified, and the total number of samples represents the total number of samples. The range of values ​​for this indicator is... A higher value indicates better overall classification performance of the model.

[0206] Finally, the action classification module integrates the classification results of four models (ResNet, SVM, RF, and XGB), and determines the optimal model output as the final action category through accuracy evaluation. Simultaneously, for actions involving structural interaction such as "climbing" and "leaning," spatial consistency correction is further performed using scaffolding segmentation masks to correct potential misclassifications. This module ultimately outputs the action category result for each person. This result is not only used for worker safety behavior analysis but also serves as the basis for action risk indicators in the subsequent risk assessment module.

[0207] Step 5: Calculation and comprehensive assessment of risk indicators.

[0208] After completing detection, identification and feature extraction, this invention constructs a multi-index system for risk assessment of high-altitude workers, and uses the Analytic Network Process (ANP) and ELECTRE III multi-criteria ranking method to achieve comprehensive assessment and ranking.

[0209] 1) Risk indicator system;

[0210] Based on the needs of engineering safety management and expert advice, four core risk indicators were selected:

[0211] (1) Action risk The categories output by the action classification module are mapped to risk scores (climbing = 5, leaning = 4, squatting = 3, sitting = 2, standing = 1). For the "climbing" and "leaning" categories, if the overlap rate between the skeleton keypoints and the scaffolding segmentation mask is lower than the threshold... If so, the risk score will be reduced by one level.

[0212] (2) PPE Wearing The YOLOv8 PPE system is used to assess whether helmets, vests, and seat belts are worn. Wearing both earns 1 point, wearing only one earns 3 points, and not wearing any earns 5 points.

[0213] (3) Working height The level is determined by the relative vertical position of the personnel detection frame within the scaffolding detection frame: bottom layer = 1 point, middle layer = 3 points, top layer = 5 points.

[0214] (4) Structural stability : Boundary overlap rate between scaffolding segmentation mask and standard profile As an indicator of completeness, 1 point is awarded for over 80%, 3 points for around 60%, 4 points for 20-40%, and 5 points for less than 20%.

[0215] 2) Indicator normalization;

[0216] Different indicators have different dimensions and ranges, so they need to be normalized first. The interval is normalized using Min-Max linear normalization:

[0217]

[0218] in, For the first The object in the first The original values ​​of each indicator These are the minimum and maximum values ​​of the indicator, respectively. This is a normalized result. Qualitative grades can be directly mapped at equal intervals according to the scores.

[0219] 3) ANP weight calculation;

[0220] ANP organizes the indicators into a network structure, allowing for interdependencies. First, pairwise comparison judgment matrices are constructed. :

[0221]

[0222] in, As an indicator The relative weights. The judgment values ​​are obtained by expert scoring based on the Saaty 1–9 scale.

[0223] The weight vector is obtained through normalization and the eigenvector method:

[0224]

[0225] Consistency ratio Used to verify the reasonableness of expert judgment:

[0226]

[0227] in, As a random consistency indicator, The judgment matrix is ​​acceptable. The final weights are calculated using the ANP hypermatrix to reflect the dependencies between indicators.

[0228] 4) ELECTRE III Comprehensive Sorting;

[0229] ELECTRE III achieves multi-object sorting by constructing consistency and discrepancy matrices.

[0230] (1) Consistency Index: For personnel The consistency index is defined as:

[0231]

[0232] in, Indicates the tolerance threshold Internal, indicators support" Superior The set of " The weights are the indicator weights.

[0233] (2) Inconsistency index:

[0234]

[0235] in, As a veto threshold, On this indicator Clearly superior The set of indicators.

[0236] (3) Total Consistency Index:

[0237]

[0238] (4) Distillation order: based on Construct a graph showing the relationship between superior and inferior characters, and obtain the final comprehensive ranking vector through ascending and descending distillation. .

[0239] 5) Output the results;

[0240] The final output includes:

[0241] (1) Overall risk score for each person ;

[0242] (2) Risk ranking arranged in descending order of score;

[0243] (3) Consistency test between model ranking and expert ranking (using Kendall Jenner) coefficient).

[0244] This method combines the objectivity of ANP's weights with the fuzziness tolerance of ELECTRE III, and can still obtain a reasonable risk ranking when there are some inconsistencies in the judgments.

[0245] Example 2:

[0246] To verify the effectiveness of the proposed framework, this invention systematically trained models for each submodule and conducted comprehensive tests on multiple actual construction videos. The experimental procedure included: first, training and validating YOLOv8 object detection, SAM2 scaffolding segmentation, YOLO-Pose pose estimation, and dual-path motion classification models based on self-built and publicly available datasets; then, applying the trained models to real video data to identify and analyze the safety helmet wearing status, action categories, working height, and scaffolding structural integrity of construction workers; finally, calculating risk indicators based on these indicators and using ANP and ELECTRE III methods to conduct comprehensive risk assessment and ranking of different workers.

[0247] Step 1: Train the model for each sub-module to obtain the risk assessment model for high-altitude workers.

[0248] 1) YOLO object detection;

[0249] First, the YOLOv8 model is used to detect people and related targets in the images. This invention selects the medium-sized YOLOv8 model (YOLOv8m) released by Ultralytics, which achieves a good balance between accuracy and speed. Detected targets include people, scaffolding, PPE (hard hats, vests, safety belts), and scaffolding itself. To train the detection model, this invention constructs a labeled dataset containing the above targets: the people and PPE portions utilize the publicly available Roboflow ConstructionSite Safety image dataset (original 2210 images, 2000 retained after filtering and cleaning), covering various construction environments, lighting conditions, and viewpoint variations.

[0250] The scaffolding target data was obtained from high-altitude operation photos collected by this invention, resulting in 258 images with obvious scaffolding structures. All images were bounding boxes annotated using the LabelMe tool, and the training and validation sets were divided in an 8:2 ratio. To address the relatively small number of scaffolding category samples, this invention employs an oversampling strategy during training to improve the model's ability to identify this category.

[0251] The training process is based on the default loss function settings of YOLOv8, including a weighted sum of confidence loss, classification loss, and CIoU bounding box loss. The optimizer used is AdamW, with an initial learning rate of 0.001, a batch size of 16, and 100 training iterations. This invention applies data augmentation techniques such as Mosaic and MixUp to improve the model's generalization ability. During training, after each iteration, the mAP metric is calculated on the validation set, and the weights of the best-performing model are saved. Ultimately, the YOLOv8 detection model can stably identify construction workers and their PPE, as well as the location of scaffolding structures in the image. The personnel bounding boxes output by this module will be used as cropped inputs for subsequent pose estimation and action classification, while the detection results of safety helmets and safety belts will directly serve as the basis for PPE wearing indicators in risk assessment.

[0252] 2) SAM2 scaffolding semantic segmentation;

[0253] To obtain information about the supporting structure where personnel are located, this invention designs a scaffolding region segmentation module. Based on the Segment Anything Model proposed by Meta, this invention selects its improved version, SAM2, as the backbone of the segmentation model. SAM2 is a cue-driven segmentation network that can segment corresponding target regions in an image based on cues such as points and bounding boxes. This invention utilizes this characteristic, allowing the aforementioned detection module to provide a rough bounding box of the scaffolding as an initial cue, and then randomly generating several point cues to guide the model to refine the segmentation. In specific implementation, this invention freezes the backbone encoding network of the SAM2 model and only fine-tunes its mask decoder and cue embedding layer to adapt to the special texture and shape features of the scaffolding structure.

[0254] The training data consists of the detected scaffold frame cropped image and the corresponding manually labeled mask. This invention uses Photoshop to draw a binary mask based on the actual scaffold outline, ensuring that structures such as thin rods and cross supports are fully marked. Then, a script is used to convert the RGB to a single-channel grayscale image to accommodate the fine-tuning requirements of the sam2 image, such as... Figure 5 As shown, a total of 400 pairs of image and mask sample datasets were created for fine-tuning training, including complex cases such as high-angle shots, partial occlusion, and blurred edges, to enhance the robustness of the model.

[0255] During training, for each image, this invention randomly generates 5-10 mixed positive and negative cue points (some located on the scaffolding, some in the background), which are input into the SAM2 model to predict the mask. The loss function is a weighted sum of binary cross-entropy and mask IoU loss. The optimizer is Adam, the learning rate is set to 5e-5, the batch size is 4, and the training lasts for 30 epochs. To improve segmentation stability, this invention employs multi-point cue fusion during the inference phase, that is, sampling cue points multiple times for the same image and merging the masks to reduce the impact of randomness in a single cue.

[0256] To improve the accuracy of scaffolding segmentation prediction and avoid interference from irrelevant backgrounds, this module integrates with the YOLO detection module. First, the YOLO scaffolding detection results are cropped before scaffolding segmentation. The process is as follows: Figure 6 As shown.

[0257] 3) Human posture estimation;

[0258] This invention employs the YOLO-Pose model for human skeleton detection. A total of approximately 1800 images containing human figures were collected, with 80% used for training and 20% for validation. During training, the weights of the YOLO backbone were fixed, and only the newly added keypoint branches and some shared layers were trained to prevent gradient interference from the detection task. Training lasted 120 epochs with a learning rate of 0.0005 and a batch size of 32. Data augmentation techniques such as randomized affine transformations and horizontal flipping were applied to improve the model's adaptability to different poses and viewpoints.

[0259] The coordinates and confidence scores of each human keypoint are saved in JSON format for use by the downstream motion classification module. To ensure keypoint quality, this invention filters or interpolates keypoints with confidence scores below a threshold (e.g., 0.3) to reduce the interference of missing keypoints on classification. After the above processing, the pose estimation module can stably provide a relatively complete human skeleton structure, providing a basis for complex motion classification.

[0260] 4) Classification of high-risk actions;

[0261] After acquiring the human skeleton, this invention needs to determine whether the worker is in a high-risk movement state. This invention designs a dual-path action classification strategy: one is skeleton image classification based on deep learning, and the other is pose parameter classification based on manually generated features. The two methods complement each other and are used to compare and analyze which method is more effective for different movements.

[0262] (1) ResNet classification of skeleton images: The script in this invention visualizes the 17-point skeleton output from the previous step, marking each joint with red dots and connecting them with red lines to form a skeleton diagram, such as... Figure 7 As shown. The background is set to pure white to highlight the human posture structure. For images containing multiple people, this invention individually crops the people identified by Yolo-Pose and then performs skeleton extraction, as shown. Figure 8 As shown. This method is applied to both training set creation and prediction. All skeleton images are uniformly scaled to 224×224 pixels and used as image input to a pre-trained ResNet-18 convolutional neural network. This invention adds a fully connected layer to ResNet to output 5 categories (corresponding to standing, sitting, squatting, climbing, and leaning), and trains for 80 rounds using cross-entropy loss, with an initial learning rate of 0.001 and a batch size of 64. The training and test sets are randomly divided based on skeleton images (corresponding to pose estimation data).

[0263] (2) Traditional classification of posture features: Three commonly used machine learning models, SVM, Random Forest, and XGBoost, were selected for modeling. In feature design, considering that the key to distinguishing actions lies in the relative positional changes of different parts of the body, this invention extracted 18-dimensional geometric features, including: the angles between major joints such as left and right hip-knee, knee-ankle, and shoulder-hip; multiple sets of upper / lower limb length ratios; the position of the body's center of gravity relative to height; the degree of squatting by comparing the height of the left and right knee joints; the center of gravity calculated from the average coordinates of each key point; and the body's width-to-height ratio. All features were normalized for each individual to eliminate the influence of individual body size differences. The feature extraction process was automated by writing a dedicated script, and the extraction results were saved as a CSV file, providing standardized feature input for training the action classification model. Feature vector It will be used as input to train three traditional classifiers: SVM, Random Forest (RF), and XGBoost (XGB).

[0264] Table 2. Examples of SVM / RF / XGB training set CSV files

[0265]

[0266] The classifier is then trained using 80% of the samples and tested using 20%, with 5-fold cross-validation used for parameter tuning to prevent overfitting. For SVM, this invention uses an RBF kernel and a grid search penalty coefficient C and kernel parameter γ; for RF and XGB, this invention mainly adjusts parameters such as the number of trees and maximum depth. During training, samples with excessively low keypoint detection confidence (below 0.3) resulting in significantly incomplete skeletons are removed to ensure the model learns reliable pose patterns.

[0267] (3) Structure-Assisted Judgment Fusion: Addressing the limitations of both depth-based and traditional classification methods, this invention further designs a rule-based fusion strategy, specifically designed to improve the reliability of recognizing leaning and climbing actions. Specifically, this invention utilizes a scaffolding segmentation mask to provide spatial reference: when the ResNet model determines an action as "climbing" or "leaning," this invention checks whether key points of the human skeleton (especially the shoulder and back positions) overlap with the segmented scaffolding area. If the upper body of the person is detected to be pressed against the scaffolding, this invention considers the model's judgment valid; conversely, if the skeleton is completely suspended and not in contact with the structure, it may be a misjudgment, and the action should be downgraded (e.g., "climbing" should be corrected to "standing"). This invention sets a pixel overlap threshold of 15 to filter out noise cases where only edges are in contact.

[0268] The risk assessment model for high-altitude workers was obtained through the above training process. The model framework includes a multi-category target detection module, a scaffolding area segmentation module, a posture estimation module, and a high-risk action classification module.

[0269] Step 2: Apply the trained model to real video data to identify and analyze construction workers.

[0270] This embodiment aims to verify the effectiveness of the high-altitude worker risk assessment framework based on computer vision and multi-criteria decision fusion proposed in this invention. Through the analysis of two typical video scenarios, the applicability and robustness of the proposed method under different levels of complexity are demonstrated. Scenario 1 is a two-person work scenario, focusing on demonstrating the computational process of ANP–ELECTRE III; Scenario 2 is a five-person work scenario, verifying the model's scalability under multi-person work conditions. All video frames are from real construction sites. After processing by YOLOv8 target detection, YOLO-Pose pose estimation, SAM2 scaffolding segmentation, and PPE detection modules, information such as the worker's action status, PPE wearing status, working height, and structural integrity is obtained.

[0271] The video frame image recognition results are as follows:

[0272] Figure 9 and Figure 10 The results of recognition for two video scenes are shown. The model can simultaneously output bounding boxes, skeleton key points, scaffolding segmentation masks, and PPE classification labels, achieving multi-information fusion recognition.

[0273] In scenario 1, two workers were detected: worker A was standing on the middle scaffolding platform, wearing full PPE; worker B was climbing on the top floor, not wearing a safety belt, and the risk of his actions was significant.

[0274] In scenario 2, five workers were identified: Worker A was on the left side of the top floor, standing, with his safety helmet visible but his safety belt not detected; Worker B was on the middle platform, wearing complete PPE; Workers C and D were both on the right side of the top floor, climbing, with some PPE missing; Worker E was on the ground, standing, with good protection.

[0275] This process reveals that computer vision models can simultaneously perform multi-target detection, structure segmentation, action recognition, and protective equipment detection within a single image frame, and visualize multi-source information in a unified manner. This provides an intuitive and reliable foundation for subsequent risk quantification and ranking analysis.

[0276] Step 3: Calculate risk indicators and use the ANP and ELECTRE III methods to conduct a comprehensive risk assessment and ranking of different workers.

[0277] 1) Determining the weights of the ANP criterion;

[0278] First, five experts with experience in construction safety management were invited to conduct pairwise comparisons of the four criteria. The experts concluded that action risk was the most critical factor in the overall risk assessment, followed by working height, then PPE condition, while structural integrity was of relatively lower importance. The comprehensive scoring results yielded a judgment matrix:

[0279]

[0280] The weights of each indicator are obtained by calculating the weights using the eigenvector method:

[0281]

[0282] Among them, action risk (46.7%) had the highest weight, followed by working height (27.7%).

[0283] The consistency test results are as follows: This indicates that the judgment matrix has good consistency.

[0284] 2) Values ​​for worker risk indicators;

[0285] Based on the detection and segmentation results, this invention quantifies various risk factors for worker A and worker B. The action risk value is derived from the outputs of the YOLO-Pose and ResNet classifiers; the PPE status is determined by YOLO detection of the safety helmet and safety belt; the working height is based on a hierarchical mapping of the worker's bounding box position within the scaffolding segmentation result; and structural integrity is determined by the consistency between the output of the SAM2 segmentation module and the boundary of the original image. If the segmentation result shows local missing or occluded sections of the scaffolding, the risk value at the corresponding location increases. Risk values ​​are assigned to seven workers (A and B in scenario 1; A–E in scenario 2), and utility values ​​(u = 1 - risk) are calculated. The definitions of each indicator are as follows:

[0286] C1 Motion Risk: Climbing 0.85, Standing 0.40;

[0287] C2 PPE status: Complete 0.10, Partially Missing 0.30, Severely Missing 0.70;

[0288] C3 working height: ground level 0.10, middle level 0.50, top level 0.90;

[0289] C4 Structural Integrity: Based on SAM2 mask integrity, the integrity region is set to 0.25-0.30.

[0290] Table 3 summarizes the risk and utility values ​​for each worker.

[0291] Table 3 Risk and utility values ​​for each worker

[0292]

[0293] 3) ELECTRE III sorting calculation;

[0294] To address uncertainty, this invention employs the ELECTRE III transcendental ranking method. An imperceptibility threshold is set for each criterion. Preference threshold veto threshold .

[0295] (1) Calculation of local coordination degree:

[0296] The definition of local coordination degree is:

[0297]

[0298] in, It has poor utility.

[0299] Take action risk (C1) as an example:

[0300]

[0301] Similarly, the local coordination matrix of each indicator was calculated. The results show that the standing workers performed better overall than the climbing workers.

[0302] (2) Calculation of overall coordination degree:

[0303] Based on the ANP weights, the overall coordination degree is:

[0304]

[0305] For example, in scenario 1:

[0306]

[0307] This indicates that worker A is superior to worker B in all criteria.

[0308] (3) Rejection test and credibility:

[0309] If a certain criterion differs If so, the corresponding direction is rejected. In this case: the largest difference is the PPE criterion ( If the threshold is equal to 5, no veto is triggered; all others are less than 5. Therefore, no veto occurred.

[0310] Credibility is defined as:

[0311]

[0312] in,( ) represents all pairs of ( This constitutes a set of opposing criteria.

[0313] Taking scenario 1 as an example, the utility values ​​of workers A and B are:

[0314]

[0315] Substituting the overall coordination degree (C(A,B)=0.9046), we get:

[0316]

[0317] This indicates that worker A's safety is significantly better than worker B's.

[0318] In multi-person scenarios, pairwise comparisons between all workers are calculated in the same way, and the results are used to construct a confidence matrix and perform the final ranking accordingly.

[0319] 4) Calculation results and ranking analysis;

[0320] The risk ranking of the seven workers was obtained through pairwise comparisons and net flow calculations using ELECTRE III, as shown in Table 4.

[0321] Table 4 Risk Assessment Results

[0322]

[0323] The final sort is:

[0324]

[0325] As shown in Table 4, the ranking results are consistent with the actual working conditions in the video: the risk is lowest for ground and mid-level workers, while the risk is significantly higher for climbers and those working at higher levels. Secondly, ELECTREIII effectively handles conflicts between indicators in multi-person scenarios, showing greater stability than the weighted average method. Furthermore, although the scaffolding structural integrity varies little in this example, it still retains an influence path in the model; once structural damage or occlusion is detected, the weight of this item will be significantly increased. Finally, the model output results are consistent with the Kendall assessment by five experts. The coefficient of 0.79 indicates good consistency in the ranking, verifying the reliability and interpretability of the model.

[0326] Overall, the results of this embodiment demonstrate that the proposed vision-decision fusion framework can effectively integrate multi-indicator risks in real-world scenarios and maintains a high degree of consistency with expert judgment in output ranking. This not only verifies the feasibility of the method but also provides a reference for intelligent safety supervision of actual construction sites.

[0327] The above description is only for the purpose of helping to understand the method and core essence of the present invention, but the scope of protection of the present invention is not limited thereto. For those skilled in the art, any equivalent substitutions or modifications made to the technical solution and inventive concept disclosed in the present invention within the scope of the technology disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for automatic identification and assessment of construction risks for workers operating high-altitude scaffolding, characterized in that, Includes the following steps: S1. Multi-category target detection: Construct a target detection model to detect and locate workers, personal protective equipment, and scaffolding structures from the original monitoring images; S2. Scaffolding region segmentation: Based on the location markers of the scaffolding structure in the monitoring image, an image segmentation network is used to generate a scaffolding segmentation mask to obtain the scaffolding region segmentation result; S3. Human pose estimation: Based on the positioning marks of workers in the monitoring images, the key points of the human skeleton are detected by the human pose estimation model. S4. High-risk action classification: Design a dual-path action classification strategy, including deep learning-based skeleton image classification and pose parameter classification based on artificial features. Combine the key points of the human skeleton to determine the action category. Compare and analyze the dual-path action classification results to evaluate the accuracy of the action classification results. In addition, design a structure-assisted discrimination fusion strategy, using the scaffolding region segmentation results to provide spatial reference, check whether there is pixel overlap between the key points of the human skeleton and the scaffolding region, and correct the action classification results. S5. Risk indicator calculation and comprehensive assessment; The risk index system is constructed by selecting the degree of action risk, the degree of wearing of personal protective equipment, the working height, and the stability of the scaffold structure as core risk indicators. The core risk indicators are normalized, ANP weights are calculated, and a comprehensive vector is calculated to obtain the ELECTRE III comprehensive ranking.

2. The method for automatic identification and assessment of construction risks for workers operating high-altitude scaffolding according to claim 1, characterized in that, The target detection model in step S1 is as follows: The target detection model is built on the YOLOv8 architecture; Given an input image The target detection model first extracts multi-scale feature maps through a feature extraction network; Subsequently, the feature fusion module aggregates the multi-scale features, and the detection head outputs a set of... Prediction results: in, This indicates the center coordinates and dimensions of the predicted bounding box. This is the class probability vector of the predicted bounding box. The probability of the existence of the target; The set of detection categories for the object detection model is as follows: in, This indicates the test results from the operators. This indicates the test results for personal protective equipment. This indicates the inspection results of the scaffolding structure.

3. The method for automatic identification and assessment of construction risks for workers operating high-altitude scaffolding according to claim 1, characterized in that, The image segmentation network in step S2 is as follows: The image segmentation network is built based on the SAM2 framework; During training, the image backbone encoder of SAM2 is frozen, and only the parameters of the mask decoder and cue embedding layer are fine-tuned; and the optimization objective during training consists of a weighted sum of binary cross-entropy loss and mask IoU loss. A multi-point cue fusion strategy is introduced into the trained and optimized image segmentation network for segmentation: The candidate bounding boxes of scaffolding structures output by the object detection model are used as the initial cue boxes for the image segmentation network, and cue boxes are randomly sampled inside and outside the candidate bounding box regions to form a set. ; For the same input image, generate Group random point prompt The masks were predicted respectively. Final output: in, This is the final probability mask after fusion.

4. The method for automatic identification and assessment of construction risks for workers operating high-altitude scaffolding according to claim 3, characterized in that, The human pose estimation model in step S3 is as follows: The human pose estimation model is constructed by adding a keypoint prediction branch to the traditional YOLO detection framework.

5. The method for automatic identification and assessment of construction risks for workers operating high-altitude scaffolding according to claim 3, characterized in that, The deep learning-based skeleton image classification in step S4 is as follows: ResNet-18 is used as the base model for skeleton image classification. The coordinates of 17 key points output by the human pose estimation model are visualized, each key point is marked and connected to form a skeleton image. The skeleton image is then input into the ResNet-18 network for classification, including five types of actions: standing, sitting, squatting, climbing, and leaning. The action classification results are then output.

6. The method for automatic identification and assessment of construction risks for workers operating high-altitude scaffolding according to claim 5, characterized in that, The pose parameter classification based on artificial features in step S4 is as follows: The posture feature vector is calculated using human joint coordinates. The joint angle features, trunk principal axis direction angle, shoulder width, height-to-width ratio, center of gravity position, and symmetry score are calculated respectively. The joint angle features include the angle between the left and right upper limbs formed by elbow-shoulder-wrist, the angle between the left and right lower limbs formed by hip-knee-ankle, the angle between the left and right shoulders formed by elbow-shoulder-hip, and the left and right trunk bending angle formed by shoulder-hip-knee. All features are normalized and combined to form the posture feature vector. The pose feature vectors are classified using traditional machine learning models, which may include one or more of the following: SVM model, Random Forest (RF) and XGB classifier.

7. The method for automatic identification and assessment of construction risks for workers operating high-altitude scaffolding according to claim 6, characterized in that, In step S4, the check for pixel overlap between key points of the human skeleton and the scaffolding area is as follows: Let the set of key points output by the human pose estimation model be: in, For the first Pixel coordinates of key points; Overlap determination is expressed by the following rules: in, For index functions, The final mask for the scaffolding area. The threshold for pixel overlap; if it satisfies Then it is determined that there is an interaction relationship between the key point and the scaffolding area; If the key point overlaps with the scaffolding area, the action category is valid; otherwise, it is a potential misjudgment.

8. The method for automatic identification and assessment of construction risks for workers operating high-altitude scaffolding according to claim 1, characterized in that, The risk indicator system in step S5 is as follows: The risk level of an action is mapped from the action category output by the high-risk action classification to the corresponding risk score. The degree of personal protective equipment (PPE) wearing is determined by the detection results of PPE in the target detection model. PPE includes safety helmets, vests, and seat belts, and the number of items not worn is mapped to the corresponding risk score. The working height is determined by the relative vertical position of the worker detection box in the target detection model within the scaffold detection box, and is then mapped to the corresponding risk score. The stability of the scaffold structure is mapped to the corresponding risk score by the boundary overlap rate between the scaffold segmentation mask and the standard profile.

9. The automatic identification and assessment system for construction risks of high-altitude scaffolding workers obtained by the method according to any one of claims 1-8, characterized in that, include: The multi-category target detection module uses the YOLOv8 model to detect and locate workers, personal protective equipment, and scaffolding structures in the input raw monitoring images, and outputs predicted bounding boxes for workers, personal protective equipment, and scaffolding structures. The scaffolding region segmentation module is designed based on the SAM2 framework and a multi-point cue fusion strategy. It is used to generate a scaffolding segmentation mask by using the predicted bounding boxes of the scaffolding structure output by the multi-class object detection module as the initial cue boxes. The pose estimation module uses the YOLO-Pose model, which is built by adding a key point prediction branch to the traditional YOLO detection framework. The predicted bounding boxes of the workers output by the multi-class target detection module are clipped and input into the YOLO-Pose model. The YOLO-Pose model outputs 17 key points of the human skeleton. The high-risk action classification module includes a skeleton image classification model based on ResNet-18 and a pose parameter classification model based on SVM, Random Forest (RF), and XGB classifiers, respectively. Both the skeleton image classification model and the pose parameter classification model combine the key points of the human skeleton to determine the action category. The action category output by the model with the highest accuracy is taken as the action classification result. Furthermore, the action classification result is corrected by checking whether there is pixel overlap between the key points of the human skeleton and the scaffolding segmentation mask. The risk indicator calculation and comprehensive assessment module establishes a risk assessment system, combining ANP weight calculation with ELECTRE III ranking to quantitatively rank the risk status of multiple workers.

Images