A comprehensive identification method for unsafe behaviors of construction site personnel

Abstract

The present invention discloses a comprehensive identification method for unsafe behaviors of construction site personnel, which specifically relates to the field of construction site image analysis and identification, including: constructing a boundary segment connection diagram through edge detection and Hough transform, and generating a risk mask area based on plane projection trajectory and buffer zone; performing instance segmentation and posture estimation on personnel, extracting joint points and calculating center of gravity points, screening key points according to operation type and constructing a relative vector sequence; extracting the angle between the main posture axis and the boundary normal of continuous frames, generating a direction consistency matrix and an outward-inclined behavior mark; finally, identifying the resident section within the risk mask area and outputting an oppressive illegal approach behavior label; realizing a "spatial oppression" judgment mechanism based on image feature recognition, ensuring early identification and risk warning of unsafe edge approach behaviors that exist when construction personnel perform different types of operations.

Description

Technical Field

[0001] The present invention relates to the technical field of construction site image analysis and recognition, and more specifically, to a comprehensive recognition method for unsafe behaviors of construction site personnel. Background Art

[0002] At construction sites, workers often work close to the edge of structures, whether at height, near the edge of a building, on the edge of a bridge, or on scaffolding platforms. While these areas are typically protected by guardrails and safety ropes, in practice, actions like clearing debris, turning to carry objects, and standing near the edge often lead workers into the "edge buffer zone" or even work close to the edge, creating a serious fall hazard. Traditional identification systems rely primarily on physical boundary judgments, such as whether a worker has crossed a safety line or exceeded the edge of a platform. However, these systems overlook a critical and high-risk phenomenon: even if the worker hasn't crossed the boundary, their center of gravity may be close to the boundary, or their posture may be leaning outward, creating a "pressure position" near the edge. Because these behaviors haven't resulted in a fall and haven't triggered clear warning lines, identification systems often interpret them as "normal," even though they are actually extremely risky.

[0003] Furthermore, different types of workers have significantly different movements when approaching edges. For example, lashing workers need to bend frequently, while cleaners may stand sideways, further increasing the difficulty of identification. Current image recognition systems generally lack the ability to model "edge pressure situations" and are unable to determine whether a person is in a dangerous state of pressure based on dimensions such as posture, orientation, movement amplitude, and dwell time. This technical blind spot directly affects the ability to early identify unsafe behaviors in places such as high-altitude construction areas, bridge edges, and cantilevered structures. There is an urgent need to develop a "spatial pressure" judgment mechanism based on image features to achieve early identification and risk warning of unsafe edge-approaching behaviors during construction workers' operations.

[0004] In order to solve the above problems, a technical solution is now provided. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a comprehensive identification method for unsafe behaviors of construction site personnel to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] S1: Perform edge detection on the construction site monitoring image and extract edge pixel points;

[0008] S2: Construct a boundary segment connection graph through Hough transform and spatial aggregation to delineate the boundaries of the adjacent candidate regions;

[0009] S3: Construct a monocular projection mapping structure based on the boundary of the edge candidate area, establish a buffer zone according to the planar projection trajectory of the boundary segment, and generate a spatial risk mask area coverage map;

[0010] S4: Perform instance segmentation and pose estimation on the operator in the surveillance image, extract the coordinate sequence of the human joint nodes and calculate the center of gravity of the operator's individual movement;

[0011] S5: Filter key points based on the pre-built mapping relationship between task type and posture structure, and output the key point relative vector sequence with the center of gravity as the reference origin as the posture space expression;

[0012] S6: Extract the main posture axis vector in the continuous frames from the relative vector sequence of the key points, calculate the angle between the main posture axis and the boundary normal with the normal vector of the adjacent contour line as the reference axis, and generate the direction consistency matrix and the outward tilt behavior mark;

[0013] S7: Identify the resident segments in the spatial risk mask area coverage map and output the oppressive violation proximity behavior label.

[0014] In a preferred embodiment, in S1, edge detection is performed on the construction site monitoring image, and edge pixel point sets are extracted, which specifically include:

[0015] The construction site monitoring images acquired at the construction site are smoothed using a Gaussian kernel and converted into single-channel brightness images;

[0016] The Canny edge detection operation consisting of a double-threshold gradient operator is applied to the image, and the boundary pixel set consisting of the floor, scaffolding and guardrail line is retained. All edge pixel point sets are stored in the form of Cartesian coordinates.

[0017] In a preferred embodiment, in S2, constructing a boundary segment connection graph through Hough transform and spatial aggregation, and delineating the boundary of the adjacent candidate region specifically includes:

[0018] Project the Cartesian edge pixel point set into the parameter space, perform Hough space transform through the accumulator array, and calculate the angle and displacement parameters of all linear edge distributions;

[0019] Identify high-density voting units whose continuous responses exceed a set density threshold and derive the line segment expression under the corresponding image coordinates;

[0020] In the set of straight line segments obtained by projection, the geometric center positions and directions of the line segments are spatially density aggregated, and a line segment connection graph is constructed using Euclidean distance constraints and angular similarity constraints. The subsets in the connection graph whose average length and overlap exceed the set threshold are defined as the boundaries of the candidate edge regions.

[0021] In a preferred embodiment, in S3, a monocular projection mapping structure is constructed based on the boundary of the edge candidate region, a buffer zone is established according to the plane projection trajectory of the boundary segment, and a spatial risk mask area coverage map is generated, which specifically includes:

[0022] Based on the corresponding coordinate data of the boundary segments of the identified edge candidate areas, a monocular projection mapping structure is constructed between the two-dimensional image coordinates and the actual site physical space;

[0023] The linear least squares method is used to regress the plane projection trajectory of the boundary line pixel points in the physical space, and each boundary line segment is mapped to the construction plane. The length and relative orientation offset of the linear segment at the actual scale are calibrated.

[0024] Based on the regressed boundary line trajectory, a buffer zone extension structure is constructed in the same direction as the boundary normal line, and each line segment is extended along the normal line direction to the preset buffer width to form a continuous parallel boundary space strip;

[0025] The buffer zone area is remapped to the image space according to pixel coordinates to generate a continuous mask structure and a polygon scanning method is used to fill and generate a bitmap-level risk area mask layer, and the spatial risk area coverage map of the corresponding frame is output.

[0026] In a preferred embodiment, in S4, performing instance segmentation and posture estimation on the worker in the surveillance image, extracting the human body joint node coordinate sequence and calculating the center of gravity of the worker's individual movement specifically includes:

[0027] Acquire real-time monitoring images of the construction site, calibrate the bounding boxes of the workers in the image and perform instance segmentation, and output the detection box coordinates and image region mask corresponding to each target individual;

[0028] Perform pose estimation network inference on the image area within the detection frame, extract the coordinate sequence of the operator's body joint nodes, represent all coordinates in two-dimensional image pixels, and output a node confidence list;

[0029] In the preset center of gravity node sequence, the center of gravity point of the individual operator's action is estimated based on coordinate averaging and weight filtering.

[0030] In a preferred embodiment, in S5, based on the pre-built mapping relationship between the operation type and the posture structure, the key points are selected, and the relative vector sequence of the key points with the center of gravity as the reference origin is output as the posture space expression, which specifically includes:

[0031] Based on the personnel position coordinates in the real-time image frame corresponding to the construction area, the task type identifier of the work area corresponding to the current frame is retrieved through the spatial mapping table;

[0032] Call the pre-built mapping relationship between job type and posture structure, retrieve the target job action skeleton node set according to job type, and output the node combination template for posture vector construction;

[0033] Filter the key point indexes that are consistent with the node combination template in the joint node coordinate sequence and remove auxiliary nodes that are not related to the current work type;

[0034] Based on the filtered key point coordinates, the key point relative coordinate structure is constructed with the center of gravity as the reference origin, and the relative vector sequence is output as the posture space expression.

[0035] In a preferred embodiment, in S6, the main posture axis vectors in the continuous frames are extracted from the relative vector sequence of the key points, the angle between the main posture axis and the boundary normal is calculated with the normal vector of the adjacent contour line as the reference axis, and the direction consistency matrix and the outward tilt behavior flag are generated. Specifically, the following steps are performed:

[0036] Perform time window segmentation on the relative vector sequence of key points, extract the main posture axis vectors formed by the key points in consecutive frames, and calculate the unit direction vector of each frame;

[0037] Taking the normal vector of the edge contour as the reference axis, calculate the angle between the main posture direction and the normal vector of each frame, record the angle offset sequence and construct the offset rate time graph;

[0038] The inter-frame offset variance is introduced to calculate the posture stability index of the current action stage, and the period when the variance is greater than the specified threshold is marked as the posture transfer segment;

[0039] Combining the inter-frame movement direction of the action center of gravity point with the consistency relationship between the main posture direction, it is determined whether the personnel movement trend is towards the risk area, and a direction consistency Boolean identification matrix is established;

[0040] During the period when the continuous angle deviation exceeds the set angle and the direction consistency Boolean matrix is true, the number of continuous frames is recorded and the segments exceeding the set time threshold are marked with an outward-leaning behavior flag.

[0041] In a preferred embodiment, in S7, identifying the resident segment in the spatial risk mask area coverage map and outputting the oppressive violation approach behavior label specifically includes:

[0042] Based on the segmentation of people in continuous image frames marked with extroverted behavior signs, a single-frame pixel-level mask is established. The total number of covered pixels within the mask buffer is counted to generate a sequence of overlapping area ratios within the frame.

[0043] Perform a sliding window averaging operation on the intra-frame overlap ratio sequence to construct a stable resident segment index table and filter out time segments where the continuous coverage ratio is higher than the set threshold;

[0044] The number of frames lasting for each high-coverage time segment is calculated, and short-term dwell segments whose duration does not reach the minimum time threshold are discarded, retaining the set of stable compression behavior segments;

[0045] A risk score calculation is performed on the oppressive behavior segment, and a ranking function is constructed based on the mean overlap ratio, dwell time, and direction deviation amplitude to output the oppressive violation proximity behavior label.

[0046] The technical effects and advantages of the comprehensive identification method for unsafe behaviors of construction site personnel of the present invention are as follows:

[0047] By extracting the boundaries of the adjacent areas in the construction site monitoring images through edge detection and Hough transform, and using monocular projection mapping to establish buffer zones and spatial risk mask areas, the risk areas of the construction site are accurately delineated. Based on instance segmentation and posture estimation, the human joint node sequence of the workers is extracted, and the skeleton node set is screened through the pre-built mapping relationship between the work type and the posture structure. The action skeleton structure vector diagram is constructed and the individual center of gravity point is calculated, which improves the accuracy of the posture expression of different work types. The angle calculation between the main posture axial vector and the boundary normal and the generation mechanism of the direction consistency matrix are introduced to realize the extraction of extroverted behavior signs and directional trend judgment. Combining the identification of the resident segment and the risk scoring in the spatial risk mask area, the label of the oppressive illegal approach behavior can be accurately output.

[0048] The method of the present invention can significantly improve the recognition accuracy of edge approach, tilting action and compression behavior at the construction site, and has high real-time performance, robustness and applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of a comprehensive identification method for unsafe behaviors of construction site personnel according to the present invention. DETAILED DESCRIPTION

[0050] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0051] Example 1, Figure 1 The present invention provides a comprehensive identification method for unsafe behaviors of construction site personnel, which includes the following steps:

[0052] S1: Perform edge detection on the construction site monitoring image and extract edge pixel points;

[0053] S2: Construct a boundary segment connection graph through Hough transform and spatial aggregation to delineate the boundaries of the adjacent candidate regions;

[0054] S3: Construct a monocular projection mapping structure based on the boundary of the edge candidate area, establish a buffer zone according to the planar projection trajectory of the boundary segment, and generate a spatial risk mask area coverage map;

[0055] S4: Perform instance segmentation and pose estimation on the operator in the surveillance image, extract the coordinate sequence of the human joint nodes and calculate the center of gravity of the operator's individual movement;

[0056] S5: Filter key points based on the pre-built mapping relationship between task type and posture structure, and output the key point relative vector sequence with the center of gravity as the reference origin as the posture space expression;

[0057] S6: Extract the main posture axis vector in the continuous frames from the relative vector sequence of the key points, calculate the angle between the main posture axis and the boundary normal with the normal vector of the adjacent contour line as the reference axis, and generate the direction consistency matrix and the outward tilt behavior mark;

[0058] S7: Identify the resident segments in the spatial risk mask area coverage map and output the oppressive violation proximity behavior label.

[0059] In S1, edge detection is performed on the construction site monitoring image, and edge pixel point sets are extracted.

[0060] Construction site surveillance images are captured by fixed-mounted cameras, typically located at the edges of construction floors or at designated security points, providing effective coverage of a wide area. The cameras are typically installed at a height of 2.5 to 5 meters, with an inclination angle between 30° and 60°. The captured surveillance images are typically in RGB format, with a resolution of 1920 × 1080 pixels and a frame rate of 20 to 30 frames per second. The resulting image data is fed into image processing as a three-dimensional matrix.

[0061] During the image preprocessing stage, the input RGB image is first grayscaled to reduce color interference and improve the consistency of edge feature extraction. A standard weighted average method is used to weight the R, G, and B channels to produce a single-channel luminance image. A Gaussian filter is then applied to the grayscale image to suppress high-frequency noise caused by varying lighting at the construction site, dust interference, and complex backgrounds. The filter kernel size is typically 5×5, with a standard deviation of 1.4 to achieve a reasonable balance between smoothing and edge preservation. The filtered output image serves as the input for subsequent edge detection.

[0062] In the edge detection stage, this embodiment adopts the Canny edge detection algorithm, which aims to extract target boundaries such as floor edges, scaffolding structures, and guardrail lines from the smoothed image. The algorithm includes four main steps: gradient calculation, non-maximum suppression, dual threshold detection, and hysteresis connection. First, the Sobel operator is used to calculate the horizontal and vertical gradient components of the grayscale image, and the gradient amplitude and gradient direction angle of each pixel are comprehensively calculated based on the gradient components in these two directions. Then, non-maximum suppression is performed on each pixel based on the gradient direction information, and non-edge points are removed to ensure that the output edge lines have high accuracy and clarity.

[0063] In order to further distinguish between real edges and noise, this embodiment adopts a dual threshold strategy in edge detection, and grades the detection results by setting a high threshold and a low threshold. Pixels above the high threshold are directly identified as "strong edge points", while pixels between the high threshold and the low threshold are identified as "weak edge points". In actual operation, the low threshold is generally 0.5 to 0.7 times the high threshold to balance the effects of edge preservation and noise suppression. For weak edge points, this embodiment introduces a hysteresis connection strategy to avoid false detection and breakage. Specifically, weak edge points connected to strong edge points are detected, retained and marked as valid edge points. Otherwise, they are eliminated. This process is particularly important in complex construction environments, because subtle features of structures such as scaffolding and guardrails may be ignored in the standard detection process. Hysteresis connection can improve the continuity and integrity of the edge. All edge pixel points retained by the Canny algorithm are stored in the form of Cartesian coordinates.

[0064] In S2, a boundary segment connection graph is constructed through Hough transform and spatial aggregation to delineate the boundaries of the adjacent edge candidate regions.

[0065] After edge detection is completed and a set of edge pixels stored in Cartesian coordinates is obtained, these pixels are projected into parameter space via a Hough transform to identify boundary structures that conform to straight line features. Specifically, each edge point is converted into a curve in parameter space. By accumulating and counting the intersections of the curves in parameter space, boundary line segments with linear features in the image are determined. To improve accuracy and reliability, an accumulator array is set in the parameter space, and all cells with votes exceeding a set threshold are identified. In this embodiment, this threshold is typically set to 50 votes or more (specifically, the threshold is set based on the array size) to ensure that the extracted line segments are sufficiently valid and significant. For all line segments that meet the criteria, they are mapped from the parameter space back to the image coordinate system, and a line segment expression with a start point and an end point is generated. All identified line segments are stored as structured data, including each line segment's start point, end point, geometric center, and direction vector. The direction vector is used to characterize the direction of the line segment, facilitating subsequent aggregation and connection operations. The following is a specific example of line segment structured data:

[0066] Line segment A (edge of the floor): starting coordinates (150,600), end coordinates (700,600), geometric center (425,600), direction vector (1,0) (indicating a horizontal line, along the positive direction of the horizontal axis), line segment length 550 pixels;

[0067] Line segment B (scaffolding crossbar): starting coordinates: (200, 700), ending coordinates: (600, 800), geometric center: (400, 750), direction vector: (0.89, 0.45) (indicating an upward tilt to the right), line segment length: 447 pixels;

[0068] Line segment C (guardrail line): starting coordinates: (100,500), end coordinates: (100,300), geometric center: (100,400), direction vector: (0,-1) (vertically downward), line segment length: 200 pixels.

[0069] In order to construct a complete boundary of the adjacent area, the identified line segment set is subjected to spatial density aggregation and directional similarity screening. First, the relative distance between the line segments is calculated based on the geometric center position of each line segment. If the geometric center distance between two line segments (for example, Euclidean distance) is less than the set distance threshold, they are marked as having an adjacency relationship. In this embodiment, the distance threshold is usually set to 20 pixels to ensure good spatial continuity between adjacent line segments. Secondly, the line segments with an adjacency relationship are analyzed for directional similarity. Directional similarity is judged by calculating the angle between the two line segments. If the angle between the direction vectors of the two line segments is less than 10 degrees, they are considered to have the same direction. Under the conditions that both adjacency and directional consistency are met, the two line segments are included in the same connection subset to establish a complete line segment connection graph.

[0070] After constructing the connectivity graph, all line segment subsets are further screened to extract valid boundary structures that meet the characteristics of the adjacent region. The specific criteria are an average segment length exceeding 100 pixels and an overlap ratio greater than 80%. If the total length and overlap of the line segments in a line segment subset meet these requirements, the subset is marked as a candidate adjacent region boundary. Identifying such line segment subsets effectively avoids misidentification of small structures or noise points and ensures that the extracted boundaries have high continuity and integrity.

[0071] In S3, a monocular projection mapping structure is constructed based on the boundary of the edge candidate region, a buffer zone is established according to the planar projection trajectory of the boundary segment, and a spatial risk mask area coverage map is generated.

[0072] A projection mapping structure is constructed from 2D image coordinates to construction plane coordinates based on the camera's installation parameters. Since surveillance cameras are typically fixed-mount monocular cameras, their position, angle, and focal length are known parameters. Based on the principle of perspective projection, a projection matrix is established to transform image pixels into physical space points. During this process, the construction plane is assumed to be a flat horizontal plane, with the point directly below the camera as the plane's origin. This mapping relationship accurately converts boundary line segments in the image to spatial segments on the construction plane, laying the foundation for subsequent buffer zone construction. After the projection mapping is completed, each boundary line segment mapped to the construction plane is precisely calibrated and described. Specifically, a linear regression method is used to fit the boundary line segment point set to obtain the optimal straight line representation that conforms to the actual site's physical structure. Each boundary line segment is represented by its start point, end point, and direction vector on the plane, and the corresponding segment length and direction are calibrated.

[0073] After obtaining the spatial representation of all boundary segments, a buffer zone construction method was further introduced. The generation of the buffer zone is based on the extension of the normal direction of the boundary segment. By constructing parallel strip areas on both sides of the boundary segment, the potential danger area can be covered. In the specific operation, the normal direction of each boundary segment is calculated based on the direction vector of each boundary segment, and the normal direction is appropriately extended. For example, for the outer edge segment of the construction floor, each segment is extended by 0.5 to 1 meter on the inside and outside to form a strip area with a certain width. The width of this strip area can be adjusted according to the safety standards of the construction site or the specific application scenario to meet different safety monitoring needs.

[0074] To ensure the continuity and integrity of the buffer zone, all generated strips were spliced and fused. By smoothing and compensating the edges of adjacent areas, multiple discrete buffer zone segments were connected into a single, continuous space. In particular, intersecting or overlapping areas were automatically merged and their edges smoothed to eliminate breaks and discontinuities caused by directional differences between segments. This process resulted in the generation of a complete, coherent risk zone strip.

[0075] After the buffer zone is constructed and merged, the generated risk area is mapped back to the image space to facilitate the generation and visualization of the risk area mask. By polygon filling and rendering the area mapped back to the image space, a risk area mask layer is generated in the form of a bitmap. The mask layer is displayed as follows:

[0076] Format: binary layer in bitmap format, the size is the same as the original image;

[0077] Mask area value: The pixel value of the risk area is 255 (white), and the pixel value of the safe area is 0 (black);

[0078] Overlay Display: Generates an image with highlight marks by overlaying the mask layer with the original image in a semi-transparent form.

[0079] In this way, high-risk areas in the construction site can be displayed in an intuitive form, providing accurate spatial reference for subsequent personnel behavior identification and violation detection.

[0080] In S4, instance segmentation and posture estimation are performed on the workers in the monitoring image, the coordinate sequence of the human joint nodes is extracted, and the center of gravity of the individual movement of the workers is calculated.

[0081] To accurately detect and demarcate multiple workers in complex scenes, deep learning models are used for object detection and instance segmentation in construction site surveillance images. Highly accurate and efficient convolutional neural networks (such as YOLOv8 or Mask R-CNN) are deployed to perform real-time inference and processing of construction site surveillance images.

[0082] Each detected worker in the image is output as a bounding box and instance mask. The bounding box provides the individual's position and size in the image, while the instance mask accurately segments the area outline of each worker. Instance segmentation distinguishes adjacent or overlapping workers, providing high-quality input data for subsequent pose estimation. Detection results are stored as structured data, including the coordinates of the top-left and bottom-right corners of each detection box, detection confidence, and instance mask matrix. Target detection and instance segmentation are performed continuously within the monitoring frame rate.

[0083] After detecting and segmenting the worker, pose estimation is performed on the image region within each detection frame to extract a sequence of joint nodes and characterize the person's movements and posture. A pose estimation model based on a deep neural network (such as OpenPose, HRNet, or PoseNet) is used to extract the corresponding image region from each detection frame and perform pose estimation inference. The model identifies multiple key nodes of the human body and outputs the 2D pixel coordinates and confidence values for each node. Typically, detected nodes include the top of the head, neck, shoulders, elbows, wrists, hips, knees, and ankles. The detection result for each node consists of a coordinate value and a confidence score, which measures the accuracy and reliability of the detection. In this embodiment, a standard structure of 17 key nodes (COCO-17 standard) is adopted, and each node is uniformly numbered and semantically labeled. The pixel coordinates of the nodes are represented as positions in the 2D image plane, and the detection results for all nodes are output as a data table.

[0084] After extracting the joint nodes of each operator, these nodes are further processed to calculate the individual's movement center of gravity. The calculation of the center of gravity is the core step for posture analysis and behavior judgment. In order to accurately calculate the center of gravity, this embodiment introduces a center of gravity calculation method based on node weighted averaging and filtering processing. The scheme presets a set of node sequences related to the center of gravity calculation, including hip, shoulder and knee positions (which can be increased or decreased according to accuracy requirements). According to the preset weights of different nodes in the human body structure, the coordinates of these nodes are weighted averaged to calculate the two-dimensional pixel coordinates of the overall center of gravity. The calculated center of gravity information is output in the form of each frame and saved together with the previously extracted joint node sequence, through dynamic analysis and trajectory evaluation of the center of gravity.

[0085] In S5, key points are selected based on the pre-built mapping relationship between the task type and the posture structure, and a relative vector sequence of key points with the center of gravity as the reference origin is output as the posture space expression.

[0086] On a construction site, different areas usually correspond to different types of work tasks. For example, high-altitude work areas may be concentrated on the outer edge of the floor and the scaffolding area, while ground operations may involve material handling and equipment installation. In order to effectively identify the behavior of workers in different scenarios, a matching mechanism between a pre-labeled (manually labeled) spatial mapping table and a work type task type identifier is introduced. The pixel coordinates of the personnel position in the real-time image frame are mapped to the physical space coordinates of the construction site. The spatial mapping table is retrieved to determine the current work area where the personnel is located, and the corresponding work type task type identifier is extracted from it. For example, when a person is detected in the area at the edge of the floor, it is identified as "high-altitude work"; when a person is detected in the material stacking area on the ground, it is identified as "handling work."

[0087] Different job types correspond to different motion and posture requirements. To more accurately describe a worker's posture, a pre-built mapping table is used to associate each job type with its most relevant motion skeleton node set. This mapping relationship is constructed based on the collection and analysis of a large number of construction site work scenes, and the most representative node combinations are extracted for different task types. For example:

[0088] Working at height (such as formwork installation, welding, etc.): Focus on the stability of the upper limbs and center of gravity, so the nodes are concentrated in the head, shoulders, elbows, wrists and hips;

[0089] Handling operations (such as material handling, equipment transfer, etc.): focus on the movement and posture changes of the lower limbs and center of gravity, so the nodes are concentrated in the hips, knees, ankles and shoulders;

[0090] Maintenance operations (such as guardrail inspection, equipment maintenance, etc.): Focus on the relative position of the hands and body, so the nodes are concentrated at the wrists, elbows, shoulders and head.

[0091] After obtaining a node combination template that matches the current task type, the previously extracted joint node sequence is screened and reconstructed. Since joint node sequences usually include detection results of multiple joint points (such as the 17 points in the COCO-17 standard), different job types have different requirements for these points. The original joint node sequence is selectively extracted based on the node index defined in the node combination template. All nodes that meet the template requirements will be retained, while nodes that are not in the template will be removed. In this process, not only is the joint point information required for the current job task retained, but the calculation and interference of invalid information are also reduced by removing irrelevant nodes.

[0092] After completing the node screening, the retained nodes are uniformly structured and expressed to construct an action skeleton structure vector graph that conforms to the current task type. After constructing the action skeleton structure vector graph, these nodes are further spatially structured to construct a relative vector sequence for the posture space expression. In order to ensure the stability and accuracy of the posture expression, the strategy of center of gravity alignment and relative coordinate structure is introduced when constructing the posture space. Specifically, the center of gravity position within each detection frame is first calculated, and the center of gravity is used as the reference origin to construct the relative coordinate structure of all key nodes. In this way, the uniform expression of the skeleton structure of the person can be maintained under different postures and actions. All relative vectors are represented in the form of two-dimensional vectors and form a complete node set.

[0093] In S6, the main posture axis vector in the continuous frames is extracted from the relative vector sequence of the key points, and the angle between the main posture axis and the boundary normal is calculated with the normal vector of the adjacent contour line as the reference axis to generate the direction consistency matrix and the camber behavior mark.

[0094] Based on the continuity of image frames, the posture data of each worker is segmented and processed according to time windows. Typically, each time window contains 5 to 10 frames to ensure sensitivity and stability to changes in movement. Within each time window, the relative vectors of all key points are extracted, and the main posture axis they constitute in space is calculated. The main posture axis is extracted by directional aggregation and weighted averaging of the key point sequence. The core nodes calibrated in the node combination template (such as the head, shoulders, hips, knees, etc.) are selected, and their average direction vector within the time window is calculated. To ensure the uniformity and robustness of the direction vector, each direction vector is normalized to obtain a unit direction vector to represent the main posture direction of the current frame.

[0095] After extracting the main posture direction for each frame, the operator's orientation is compared with the normal vector of the adjacent area to determine whether it deviates from the boundary area. Since a complete set of boundary segments and normal vectors for the adjacent candidate area has been constructed in the previous step, the posture direction and normal vector of each operator can be compared frame by frame. Using the normal vector of the adjacent contour line as the reference axis, the angle between the main posture direction and the normal vector is calculated for each frame. To ensure calculation accuracy, the angle is weighted averaged within each time window, and the time series of angle deviation is recorded.

[0096] To determine whether workers' movements during construction exhibit significant instability or sudden changes, a stability analysis method based on inter-frame offset variance was introduced. While calculating the angular offset sequence, the variance of the offset data within each time window was calculated to quantify the stability of the movement. The variance of the offset sequence within each time window was calculated and compared with a preset stability threshold. When the variance value exceeds a set threshold (based on risk identification requirements and the specific construction operation type), the current time window is marked as a "posture transition segment." This process effectively identifies sudden changes or instabilities in workers' movements and allows for real-time monitoring and recording of these high-risk sections.

[0097] After completing the posture stability analysis, the consistency judgment is further conducted by combining the movement direction of the center of gravity of the action with the main posture direction to identify whether the operator's movement trend is towards the risk area. Specifically, the center of gravity position of each frame is recorded, and the movement direction vector between adjacent frames is calculated. By comparing with the main posture direction, the directional consistency relationship between the two is determined. When the movement direction of the center of gravity is consistent with the main posture direction, it is marked as "direction consistent". To improve the accuracy of the judgment, a direction consistency Boolean identification matrix is constructed to mark the direction status of each frame. In the direction consistency matrix, if the difference in direction between the two is less than a set threshold (such as 15 degrees), it is marked as "true"; otherwise, it is marked as "false". This method can accurately determine whether the operator has a continuous movement trend towards the boundary area.

[0098] After a comprehensive analysis of directional consistency and posture deviation, the system generates an extroversion behavior flag and conducts a risk assessment based on continuity and duration. Specifically, if the deviation angle between the operator's main posture direction and the edge normal vector exceeds a set angle (e.g., 45 degrees) over a period of time, and the directional consistency matrix is "true," the operator is considered to have engaged in extroversion toward the edge area. In this case, the qualifying time period is cumulatively recorded and compared with a preset time threshold. If the duration of continuous extroversion exceeds a set time threshold (e.g., 3 seconds), an extroversion behavior flag is generated and a warning message is output. This flag serves as the basis for subsequent risk assessment and violation detection, providing real-time monitoring data for construction site safety management.

[0099] In S7, the resident segments are identified in the spatial risk mask region coverage map and the oppressive violation proximity behavior labels are output.

[0100] After performing person detection and instance segmentation on live surveillance images, a segmentation mask for each worker is generated. Based on the previous steps, the mask is a binary layer with pixel values of 255 (white) in the target area and 0 (black) in the background area. For each surveillance image frame, a person segmentation mask of the same size as the original image is output.

[0101] To identify potentially aggressive approaches, a previously constructed buffer zone mask is used as a baseline for coverage of risk areas. This buffer zone mask typically exists in bitmap form, with pixel values of 255 in risk areas and 0 in safe areas. The shape and size of the mask depend on the construction site's edge structure and the buffer zone construction method. In each frame, the worker's segmented area mask and the buffer zone mask are intersected pixel by pixel. By counting all pixels in the intersection area, the total number of pixels covered by the worker's segmented area within the buffer zone in the current frame can be calculated.

[0102] A sliding window averaging operation is performed on the coverage ratio sequence to eliminate fluctuations caused by single-frame jitter or noise. The size of the sliding window is dynamically adjusted based on the monitoring frame rate and the movement speed of the operator, usually 5 to 10 frames. Within the sliding window, all coverage ratio values are averaged to obtain a smoother and more stable coverage ratio sequence. After obtaining the smoothed coverage ratio sequence, a stable residence segment index table is further constructed. For each time segment, if the coverage ratio is continuously higher than the set threshold (specifically set according to risk identification requirements and construction operation type), it is considered that the operator has stable residence behavior in the buffer zone. For example, the coverage ratio threshold is set to 30% or higher to ensure the accuracy of the judgment result.

[0103] After completing the construction of the stable residence segment, the duration of each segment is calculated and screened. Since workers on the construction site may briefly pass through the buffer zone during movement without constituting a substantial dangerous behavior, the time threshold is set to exclude irrelevant short-term residence segments. For each high-coverage segment, the number of frames it lasts is calculated and converted into the actual time length. For example, if the frame rate of the monitoring device is 30 frames per second, a segment containing 60 frames means that the person stayed in the buffer zone for 2 seconds. This time length is compared with the preset minimum time threshold. For example, if the time threshold is set to 1 second, all segments with a stay time of less than 1 second will be discarded.

[0104] After identifying eligible residence segments, risk scores are calculated for these segments. The risk score calculation is based on a comprehensive assessment of the mean coverage ratio, residence time, and orientation deviation amplitude. A weighted summation method is used to ensure that the three parameters can accurately express a risk score value at the same scale.

[0105] Among them, the higher the coverage ratio, the larger the coverage area of the operator in the buffer zone, and the higher the risk score. At the same time, a higher risk score will be given for behaviors that stay for a long time. In addition, combined with the previously extracted angle offset between the main posture direction and the edge normal vector, the average value of the offset amplitude is calculated. If the offset amplitude is large, it indicates that the operator's posture has a tendency to tilt outward, and the risk score should be increased. The risk score value is divided into risk level intervals (low, medium, and high risks), and the oppressive violation approach behavior label is output based on the risk level. Based on the oppressive violation approach behavior label, a real-time early warning reminder of the construction personnel's behavioral risk is issued.

[0106] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters and thresholds in the formulas are set by technicians in this field according to actual conditions.

[0107] The above embodiments can be implemented in whole or in part via software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in the embodiments of this application are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0108] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0111] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, and may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0113] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0114] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0115] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A comprehensive identification method for unsafe behaviors of construction site personnel, characterized by: The steps include: S1: Perform edge detection on the construction site monitoring image and extract edge pixel points; S2: Construct a boundary segment connection graph through Hough transform and spatial aggregation to delineate the boundaries of the adjacent candidate regions; S3: Construct a monocular projection mapping structure based on the boundary of the edge candidate area, establish a buffer zone according to the planar projection trajectory of the boundary segment, and generate a spatial risk mask area coverage map; S4: Perform instance segmentation and pose estimation on the operator in the surveillance image, extract the coordinate sequence of the human joint nodes and calculate the center of gravity of the operator's individual movement; S5: Filter key points based on the pre-built mapping relationship between task type and posture structure, and output the key point relative vector sequence with the center of gravity as the reference origin as the posture space expression; S6: Extract the main posture axis vector in the continuous frames from the relative vector sequence of the key points, calculate the angle between the main posture axis and the boundary normal with the normal vector of the adjacent contour line as the reference axis, and generate the direction consistency matrix and the outward tilt behavior mark; S7: Identify the resident segments in the spatial risk mask area coverage map and output the oppressive violation proximity behavior label; In S3, a monocular projection mapping structure is constructed based on the boundary of the edge candidate region, a buffer zone is established according to the plane projection trajectory of the boundary segment, and a spatial risk mask area coverage map is generated. Specifically, the following steps are performed: Based on the corresponding coordinate data of the boundary segments of the identified edge candidate areas, a monocular projection mapping structure is constructed between the two-dimensional image coordinates and the actual site physical space; The linear least squares method is used to regress the plane projection trajectory of the boundary line pixel points in the physical space, and each boundary line segment is mapped to the construction plane. The length and relative orientation offset of the linear segment at the actual scale are calibrated. Based on the regressed boundary line trajectory, a buffer zone extension structure is constructed in the same direction as the boundary normal line, and each line segment is extended along the normal line direction to the preset buffer width to form a continuous parallel boundary space strip; The buffer zone area is remapped to the image space according to pixel coordinates to generate a continuous mask structure and a polygon scanning method is used to fill and generate a bitmap-level risk area mask layer, and the spatial risk area coverage map of the corresponding frame is output.

2. A comprehensive identification method for unsafe behaviors of construction site personnel according to claim 1, characterized in that: In S1, edge detection is performed on the construction site monitoring image, and the edge pixel point set is extracted, which specifically includes: The construction site monitoring images acquired at the construction site are smoothed using a Gaussian kernel and converted into single-channel brightness images; The Canny edge detection operation consisting of a double-threshold gradient operator is applied to the image, and the boundary pixel set consisting of the floor, scaffolding and guardrail line is retained. All edge pixel point sets are stored in the form of Cartesian coordinates.

3. A comprehensive identification method for unsafe behaviors of construction site personnel according to claim 2, characterized in that: In S2, a boundary segment connection graph is constructed through Hough transform and spatial aggregation, and the boundaries of the candidate edge regions are delineated, specifically including: Project the Cartesian edge pixel point set into the parameter space, perform Hough space transform through the accumulator array, and calculate the angle and displacement parameters of all linear edge distributions; Identify high-density voting units whose continuous responses exceed a set density threshold and derive the line segment expression under the corresponding image coordinates; In the set of straight line segments obtained by projection, the geometric center positions and directions of the line segments are spatially density aggregated, and a line segment connection graph is constructed using Euclidean distance constraints and angular similarity constraints. The subsets in the connection graph whose average length and overlap exceed the set threshold are defined as the boundaries of the candidate edge regions.

4. A comprehensive identification method for unsafe behaviors of construction site personnel according to claim 3, characterized in that: In S4, instance segmentation and posture estimation are performed on the operator in the surveillance image, the coordinate sequence of the human joint nodes is extracted, and the center of gravity of the operator's individual movement is calculated. Specifically, the following steps are performed: Acquire real-time monitoring images of the construction site, calibrate the bounding boxes of the workers in the image and perform instance segmentation, and output the detection box coordinates and image region mask corresponding to each target individual; Perform pose estimation network inference on the image area within the detection frame, extract the coordinate sequence of the operator's body joint nodes, represent all coordinates in two-dimensional image pixels, and output a node confidence list; In the preset center of gravity node sequence, the center of gravity point of the individual operator's action is estimated based on coordinate averaging and weight filtering.

5. A comprehensive identification method for unsafe behaviors of construction site personnel according to claim 4, characterized in that: In S5, key points are selected based on the pre-built mapping relationship between the work type and the posture structure, and the relative vector sequence of key points with the center of gravity as the reference origin is output as the posture space expression. Specifically, it includes: Based on the personnel position coordinates in the real-time image frame corresponding to the construction area, the task type identifier of the work area corresponding to the current frame is retrieved through the spatial mapping table; Call the pre-built mapping relationship between job type and posture structure, retrieve the target job action skeleton node set according to job type, and output the node combination template for posture vector construction; Filter the key point indexes that are consistent with the node combination template in the joint node coordinate sequence and remove auxiliary nodes that are not related to the current work type; Based on the filtered key point coordinates, the key point relative coordinate structure is constructed with the center of gravity as the reference origin, and the relative vector sequence is output as the posture space expression.

6. A comprehensive identification method for unsafe behaviors of construction site personnel according to claim 5, characterized in that: In S6, the main posture axis vector in the continuous frames is extracted from the relative vector sequence of the key points. The angle between the main posture axis and the boundary normal is calculated with the normal vector of the adjacent contour line as the reference axis. The direction consistency matrix and the outward tilt behavior mark are generated. Specifically, the following are included: Perform time window segmentation on the relative vector sequence of key points, extract the main posture axis vectors formed by the key points in consecutive frames, and calculate the unit direction vector of each frame; Taking the normal vector of the edge contour as the reference axis, calculate the angle between the main posture direction and the normal vector of each frame, record the angle offset sequence and construct the offset rate time graph; The inter-frame offset variance is introduced to calculate the posture stability index of the current action stage, and the period when the variance is greater than the specified threshold is marked as the posture transfer segment; Combining the inter-frame movement direction of the action center of gravity point with the consistency relationship between the main posture direction, it is determined whether the personnel movement trend is towards the risk area, and a direction consistency Boolean identification matrix is established; During the period when the continuous angle deviation exceeds the set angle and the direction consistency Boolean matrix is true, the number of continuous frames is recorded and the segments exceeding the set time threshold are marked with an outward-leaning behavior flag.

7. A comprehensive identification method for unsafe behaviors of construction site personnel according to claim 6, characterized in that: In S7, the resident segments are identified in the spatial risk mask area coverage map, and the oppressive violation proximity behavior labels are output, specifically including: Based on the segmentation of people in continuous image frames marked with extroverted behavior signs, a single-frame pixel-level mask is established. The total number of covered pixels within the mask buffer is counted to generate a sequence of overlapping area ratios within the frame. Perform a sliding window averaging operation on the intra-frame overlap ratio sequence to construct a stable resident segment index table and filter out time segments where the continuous coverage ratio is higher than the set threshold; The number of frames lasting for each high-coverage time segment is calculated, and short-term dwell segments whose duration does not reach the minimum time threshold are discarded, retaining the set of stable compression behavior segments; A risk score calculation is performed on the oppressive behavior segment, and a ranking function is constructed based on the mean overlap ratio, dwell time, and direction deviation amplitude to output the oppressive violation proximity behavior label.

Images