Construction safety violation behavior intelligent identification method based on unmanned aerial vehicle image

Abstract

The application relates to the technical field of image recognition, in particular to a construction safety rule violation intelligent identification method based on unmanned aerial vehicle images, which comprises the following steps: collecting construction images, generating standard images by adopting an alpha data enhancement method of source data block cyclic splicing filling, and executing double-path parallel space-to-depth conversion; constructing a non-rigid target detector containing an offset prediction branch, and adaptively matching target morphology by using an offset field; introducing a double-head mutual feedback mechanism of segmentation and attitude estimation, feeding back and correcting segmentation mask edges by using skeleton structure features, and generating instance mask and attitude heat map; constructing a joint logic based on foot distance and body invasion proportion to identify dangerous area approaching violation, and determining safety protection wearing violation by using semantic feature distance and geometric contour deviation of a correlation graph. The application realizes accurate identification of construction violations by fusing double-path image reorganization of unmanned aerial vehicle images, attitude segmentation interaction and prototype topological measurement.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to an intelligent identification method for construction safety violations based on UAV imagery. Background Technology

[0002] Construction site environments are complex and changeable, with frequent high-altitude operations and high personnel mobility, leading to frequent safety accidents. Traditional construction safety management mainly relies on manual on-site inspections and fixed camera monitoring. Manual inspections have problems such as limited coverage, low efficiency, and susceptibility to subjective factors; while fixed cameras have limitations such as many blind spots, limited viewing angles, and difficulty in covering high-altitude and dynamic operations.

[0003] Applying existing target detection and behavior recognition algorithms directly to UAV high-altitude aerial imagery faces the challenge of small and sparse target features. Since UAVs typically fly at altitudes of tens or even hundreds of meters, key targets such as safety helmets and seat belts on the ground occupy only a few pixels in the image from this perspective. The identifiable features such as texture and color contained in these small targets are extremely sparse. During continuous downsampling operations, their weak feature information is easily submerged or lost in the deep feature map of the network, resulting in serious missed detections and false detections. At the same time, the background of construction sites is extremely complex, filled with interference such as steel bars, scaffolding, and building materials. When shooting from high altitude, targets are often in dense crowds or partially obscured by other objects, which further exacerbates the difficulty of small target feature extraction and recognition.

[0004] To address this, a method for intelligent identification of construction safety violations based on UAV imagery is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent identification method for construction safety violations based on UAV imagery. The method achieves accurate identification of construction violations by fusing dual-channel imagery, performing attitude segmentation interaction, and using prototype topology measurement.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] Intelligent identification methods for construction safety violations based on UAV imagery include:

[0008] Construction images to be identified are collected by drones, and the construction images are magnified by an α data augmentation method according to the image magnification factor α to generate standard construction images. The standard construction images are then subjected to a dual-path parallel spatial-to-depth transformation using standard grids and translational misaligned grids, and are concatenated and merged in the channel dimension to generate reconstructed construction images.

[0009] A non-rigid target detector is constructed and an offset prediction branch is added to adaptively match the irregular shape of the construction target, and a target instance segmentation mask and a posture key point heatmap are generated. The target instance segmentation mask is a binary region map of the target contour, and the posture key point heatmap is a probability distribution map representing the position of key points. Based on the target instance segmentation mask and the posture key point heatmap, violations close to dangerous areas are identified.

[0010] The attitude key point heatmap is multiplied pixel by pixel with the corresponding target instance segmentation mask to generate a correlation map. Based on the correlation map, violations related to safety protective clothing are identified; and the identified construction safety violations are reported.

[0011] Preferably, the specific process of using the α data augmentation method to enlarge the construction image according to the image magnification factor α to generate a standard construction image is as follows:

[0012] The standard construction image is pre-defined to a uniform size; the original size of the construction image is obtained, the width scaling ratio and the height scaling ratio are calculated, the smaller value is selected as the image magnification factor α, and the construction image is scaled proportionally according to the image magnification factor α to obtain an intermediate image;

[0013] In the space of uniform size, a central main region and an edge filling region are defined, and the intermediate image is placed in the central main region. Returning to the unscaled original construction image, N source data blocks are randomly cropped from the original construction image. The source data blocks are interpolated and scaled to fit the size of the edge filling region. The scaled source data blocks are used to fill the edge filling region to generate the standard construction image.

[0014] Preferably, the specific process of generating the reconstructed construction image is as follows:

[0015] Set the block size and set at least two parallel reassembly paths, including a first reassembly path and a second reassembly path;

[0016] The first reconstruction path performs a space-to-depth transformation on the standard construction image to generate a first reconstruction feature map; the second reconstruction path performs a fixed spatial translation on the standard construction image, the translation offset being half the block size rounded up, and then performs the space-to-depth transformation on the translated image to generate a second reconstruction feature map.

[0017] The first reconstructed feature map and the second reconstructed feature map are concatenated along the channel dimension to generate the reconstructed construction image.

[0018] Preferably, the specific process of constructing a non-rigid target detector and adding an offset prediction branch to adaptively match the irregular shape of the construction target is as follows:

[0019] The reconstructed construction image is input into the non-rigid target detector; the non-rigid target detector extracts the depth feature map of the reconstructed construction image through a backbone network; the offset prediction branch learns and predicts the deformation offset of the target region based on the depth feature map, and generates an offset field.

[0020] The non-rigid target detector uses the offset field to perform adaptive feature sampling on the depth feature map to obtain alignment features of the irregular shape and posture of the aligned construction target; based on the alignment features, it outputs a target instance segmentation mask and a posture key point heatmap through two parallel prediction heads.

[0021] Preferably, the specific process of outputting the target instance segmentation mask and the pose key point heatmap through two parallel prediction heads is as follows: the prediction head includes a first prediction head and a second prediction head; the first prediction head generates a preliminary target instance segmentation mask based on the alignment features, and uses the preliminary target instance segmentation mask to spatially weight the alignment features to generate focusing features; the second prediction head performs localization regression based on the focusing features to generate the pose key point heatmap, and extracts the structural features of the pose key point heatmap and feeds them back to the first prediction head; the first prediction head uses the structural features to perform edge correction on the preliminary target instance segmentation mask and outputs the target instance segmentation mask.

[0022] Preferably, the specific process of identifying dangerous area proximity violations based on the target instance segmentation mask and pose key point heatmap includes:

[0023] Obtain the vector boundary of the hazardous area at the construction site; perform coordinate analysis on the posture key point heatmap to extract the coordinates of the foot key points representing the human standing position, and calculate the shortest Euclidean distance from the foot key point coordinates to the vector boundary of the hazardous area; calculate the overlapping pixel area of ​​the region enclosed by the target instance segmentation mask and the vector boundary of the hazardous area, and obtain the body intrusion ratio; construct a joint discrimination logic including the shortest Euclidean distance and the body intrusion ratio, and determine that there is a violation of approaching a hazardous area when the shortest Euclidean distance is less than a preset safety threshold or the body intrusion ratio exceeds a preset warning threshold.

[0024] Preferably, the specific process of identifying violations related to safety protective clothing based on correlation graphs includes:

[0025] The pose key point heatmap is multiplied pixel-by-pixel with the corresponding target instance segmentation mask to generate a correlation map. The high-response regions of the correlation map are used as spatial attention weights to weighted aggregate image features to generate local attribute feature vectors. The semantic feature distance between the local attribute feature vectors and the preset compliance prototype vectors is calculated, and the geometric contour deviation of the target part is calculated based on the gradient distribution of the correlation map. If the weighted sum of the semantic feature distance and the geometric contour deviation exceeds a preset threshold, a violation of safety protection wear category is determined.

[0026] Preferably, the specific process for generating the correlation map is as follows:

[0027] The pose key point heatmap and the target instance segmentation mask are unified to the same spatial scale; the target instance segmentation mask is used to perform a spatial masking operation on the pose key point heatmap to eliminate background interference signals in non-target areas; nonlinear mapping is performed on the masked image, and morphological filtering is used to remove isolated noise areas to generate the correlation map.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] 1. This application effectively solves the problems of easy loss of small target features and background noise interference caused by traditional scaling by adopting the α data augmentation method of source data block filling and combining the dual-path parallel space-to-depth conversion mechanism of standard grid and translational misaligned grid. The design uses translational misalignment to capture detailed information at the grid boundary and maintains the statistical consistency of image texture through source data block interpolation. While preserving the original high-frequency features, it avoids gradient abruptness caused by invalid filling, which significantly improves the feature extraction capability and detection accuracy of distant construction personnel and small illegal targets.

[0030] 2. This application constructs a non-rigid target detector that includes an offset prediction branch and introduces a bidirectional feature interaction and feedback mechanism between the segmented head and the pose estimation head. This enables adaptive and accurate perception of irregular shapes of construction site personnel (such as bending over, climbing, and squatting). The mechanism uses the initial segmentation mask as spatial weight to focus on key point localization, and conversely extracts the structural features of the pose skeleton to refine the segmentation edge. This effectively overcomes the problems of limb occlusion and contour adhesion caused by complex movements, and significantly improves the recognition accuracy and edge fitting of non-rigid targets in dynamic construction scenarios.

[0031] 3. This application achieves highly robust safety violation identification by constructing a dual wearing discrimination logic based on semantic feature distance and geometric contour deviation based on correlation maps, and a dangerous area judgment model that integrates foot distance and body intrusion ratio. This method abandons the traditional color threshold or rectangular box judgment that is easily affected by lighting and viewing angle, and instead uses spatial topological relationship and prototype feature measurement. It can not only accurately distinguish the physical state of standing in the danger zone and the body protruding into the danger zone, but also accurately identify the compliance of protective equipment wearing under complex lighting conditions, significantly reducing false alarms and missed alarms caused by environmental interference. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating a method for intelligent identification of construction safety violations based on UAV imagery.

[0033] Figure 2 This is a diagram of the data enhancement architecture of the present invention;

[0034] Figure 3 This is a diagram of the non-rigid target detector and dual-head interactive network structure of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1

[0037] In this embodiment, the method proposed in this invention is used to identify safety violations at construction sites, with reference to... Figure 1 , Figure 2 and Figure 3 ,include:

[0038] Construction images to be identified are collected by drones, and the construction images are magnified by an α data augmentation method according to the image magnification factor α to generate standard construction images. The standard construction images are then subjected to a dual-path parallel spatial-to-depth transformation using standard grids and translational misaligned grids, and are concatenated and merged in the channel dimension to generate reconstructed construction images.

[0039] A non-rigid target detector is constructed and an offset prediction branch is added to adaptively match the irregular shape of the construction target, and a target instance segmentation mask and a posture key point heatmap are generated. The target instance segmentation mask is a binary region map of the target contour, and the posture key point heatmap is a probability distribution map representing the position of key points. Based on the target instance segmentation mask and the posture key point heatmap, violations close to dangerous areas are identified.

[0040] The attitude key point heatmap is multiplied pixel by pixel with the corresponding target instance segmentation mask to generate a correlation map; based on the correlation map, violations related to safety protective clothing are identified; and the identified construction safety violations are reported.

[0041] The construction images to be identified by the drone are collected by using an industrial-grade multi-rotor drone equipped with a high-resolution visible light zoom gimbal camera as the collection terminal. An automatic inspection route is set in advance according to the electronic fence boundary of the construction site. The route planning adopts a bow-shaped scanning trajectory covering the entire work surface or a circular trajectory targeting key construction areas.

[0042] During the data collection process, the drone was controlled to fly at a relative height of 30 to 60 meters above the work surface to ensure that the single pixel resolution was better than 1 cm / pixel, which met the needs of subsequent small target feature extraction. At the same time, the pitch angle of the gimbal camera was set between -45 degrees and -90 degrees to take into account both vertical observation of the safety helmets on the construction workers' heads and lateral observation of their body postures.

[0043] The drone triggers shooting at preset time intervals (1 frame per second in this embodiment) to acquire original RGB construction images with a resolution of not less than 4K (3840×2160 pixels);

[0044] Specifically, the process of using the α data augmentation method to enlarge the construction image according to the image magnification factor α and generate a standard construction image is as follows:

[0045] The standard construction image is pre-defined to a uniform size; the original size of the construction image is obtained, the width scaling ratio and the height scaling ratio are calculated, the smaller value is selected as the image magnification factor α, and the construction image is scaled proportionally according to the image magnification factor α to obtain an intermediate image;

[0046] Within the uniformly sized space, a central main region and an edge filling region are defined. The intermediate image is placed within the central main region. Returning to the unscaled original construction image, N source data blocks are randomly cropped from the original construction image. These source data blocks are then interpolated and scaled to a preset basic texture unit size. When the width of the basic texture unit is less than the width of the edge filling region, the basic texture unit is repeatedly laid end-to-end along the horizontal direction. When the cumulative width exceeds the width of the edge filling region, the excess right side of the last texture unit is cropped. Similarly, when the height of the basic texture unit is less than the height of the edge filling region, it is repeatedly laid end-to-end along the vertical direction. When the cumulative height exceeds the height of the edge filling region, the excess bottom part of the last texture unit is cropped. For cases requiring simultaneous horizontal and vertical stitching, the horizontal stitching and cropping are completed first, and then the stitching result is repeatedly stitched and cropped along the vertical direction until the filling content and the pixel dimensions of the edge filling region are completely consistent, generating the standard construction image.

[0047] The uniform size of the preset standard construction image is to initialize a three-dimensional matrix container, whose height and width pixel values ​​are fixed to the input resolution values ​​required by the deep neural network model.

[0048] The calculation of the width scaling ratio and the height scaling ratio, and the selection of the smaller value as the image magnification factor α, specifically means dividing the width value of the uniform size by the original width value of the construction image to obtain the width scaling ratio, dividing the height value of the uniform size by the original height value of the construction image to obtain the height scaling ratio, comparing the size of the width scaling ratio and the height scaling ratio, and selecting the smaller value as the reference coefficient for determining the final scaling degree, that is, the image magnification factor α;

[0049] The step of scaling the construction image proportionally according to the image magnification factor α involves using a bilinear interpolation algorithm to simultaneously reduce or enlarge the length and width of the construction image according to the image magnification factor, so that the scaled image has at least one side on the long or wide side that is equal to the uniform size, and the other side is less than or equal to the uniform size, thereby obtaining the intermediate image.

[0050] The process involves defining a central main region and an edge filling region, and placing the intermediate image within the central main region. This involves calculating the pixel difference between the uniform size and the intermediate image size in the horizontal and vertical directions, dividing this pixel difference by two and rounding it down to the nearest integer to obtain the starting coordinate offset. Using this offset as the starting point, the intermediate image is copied to the very center of the three-dimensional matrix container. The remaining blank area within the three-dimensional matrix container that is not covered by the intermediate image is defined as the edge filling region. This region typically appears as a horizontal stripe located on the top and bottom sides of the image or a vertical stripe located on the left and right sides of the image.

[0051] The step of randomly cropping N source data blocks from the original construction image and interpolating and scaling the source data blocks to fit the edge-filled region specifically involves identifying the specific shape and pixel dimensions of the edge-filled region, then randomly generating one or more rectangular selection boxes within the high-resolution original construction image (which has not undergone any scaling). The position coordinates of these rectangular selection boxes are randomly generated to ensure texture diversity. Image data within these rectangular selection boxes is extracted as source data blocks. To ensure consistent texture density, the source data blocks are first normalized and scaled using bilinear interpolation to ensure their resolution density matches that of the edge-filled region. Then, the relationship between the source data blocks and the edge-filled region is determined. To address size differences in the regions, a combination of cyclic stitching and edge cropping is used for adaptation: the source data block is used as the basic texture unit and continuously laid out according to the extension direction of the edge-filled region. When the laid image range exceeds the boundary of the edge-filled region, the pixel data of the excess portion is cropped and removed. This allows the length and width values ​​of the filled pixels to be completely consistent with the edge-filled region through stitching, without changing the original geometric proportions of the pixels in the source data block. The processed pixel values ​​of the source data block are then filled into the corresponding positions of the edge-filled region, thereby eliminating black or single-color backgrounds and ultimately outputting a standard construction image that contains the complete construction scene and has realistic texture information at the edges.

[0052] By randomly cropping real texture blocks from the original high-resolution image to fill the edge regions, compared with traditional zero-fill or mean-fill methods, the overall statistical distribution consistency of the image is effectively maintained. At the same time, it eliminates the artificial high-frequency noise of hard edges caused by pixel value abrupt changes between the image content and the filled background, avoiding the convolutional neural network from extracting incorrect gradient features at the edges. Meanwhile, the real background texture of the filled image acts as a natural negative sample, forcing the network to focus more on distinguishing construction workers from complex backgrounds, thereby significantly improving the robustness of the model under non-standardized input sizes and the recognition accuracy of edge targets.

[0053] Specifically, the process of generating the reconstructed construction images is as follows:

[0054] Set the block size and set at least two parallel reassembly paths, including a first reassembly path and a second reassembly path;

[0055] The first reconstruction path performs a space-to-depth transformation on the standard construction image to generate a first reconstruction feature map; the second reconstruction path performs a spatial translation on the standard construction image, and then performs the space-to-depth transformation on the translated image to generate a second reconstruction feature map.

[0056] The first reconstructed feature map and the second reconstructed feature map are concatenated along the channel dimension to generate a reconstructed construction image;

[0057] The set block size is defined as a positive integer as the basic sampling window size, which determines the factor by which the spatial resolution is reduced and the factor by which the number of channels is increased. In this embodiment, it is set to two, which corresponds to the smallest resolvable unit of a small target at the pixel level in UAV imagery.

[0058] The first reconstruction path performs a space-to-depth conversion on the standard construction image. Without changing the total image data volume, it rearranges the pixel blocks of the standard construction image in the spatial dimension to the channel dimension according to the grid size defined by the block size. Specifically, it divides the image into multiple non-overlapping image blocks, unfolds and stacks all pixels within each image block in a fixed order, and for each size... Image blocks (in this embodiment) Extract the blocks sequentially in row-priority order from left to right and top to bottom. 1 pixel; the first image block The pixel values ​​are sequentially assigned to the first to the second pixels of the output feature map. The corresponding spatial positions of each channel (and so on) are used to fill in the blank areas at the edges after translation, with a width of [missing information] on the right and bottom of the image. Zero-value padding band of 1 pixel;

[0059] This process transforms adjacent pixels in two-dimensional space into corresponding channel values ​​in the depth direction, thereby multiplying the number of channels in the image while reducing its height and width, and outputting the first reconstructed feature map. This process preserves all high-frequency detail information of the original image without information loss.

[0060] The second reconstruction path performs a fixed spatial translation on the standard construction image. The offset of the translation is half the block size rounded down. Then, the spatial-to-depth transformation is performed on the translated image. Specifically, in order to capture features located on the sampling grid boundary of the first reconstruction path, half the block size value is first calculated and rounded down to obtain the translation step size. Using this translation step size, all pixels of the standard construction image are moved diagonally to the lower right. Zero-filling is performed on the edge blank areas generated after the movement. Then, the same spatial-to-depth transformation operation as the first reconstruction path is performed on the translated image, and a second reconstructed feature map is output. The spatial information contained in this feature map is complementary to the first reconstructed feature map in phase.

[0061] Align the spatial dimensions of the first and second reconstructed feature maps. If the dimensions are inconsistent due to translation, trim off the excess edges. Then, directly stitch the data of the two feature maps in the direction of the depth channel to generate a reconstructed construction image with double the number of channels. This reconstructed construction image contains feature information from both the original grid view and the misaligned grid view.

[0062] By setting a translational misalignment path based on half the block size, the phase complementarity principle in the sampling theorem is utilized. Traditional space-to-depth transformation will cause the target features to be segmented into non-adjacent pixel positions if the target happens to be located on the grid cutting line, resulting in feature fragmentation. This scheme uses dual-path parallel processing, and the misaligned grid of the second path can just cover the boundary center of the first path grid. This ensures that no matter where the small target (such as the head of a worker in the distance) is located in the image, it can maintain the complete local structure in one of the feature maps, thereby greatly improving the detection recall rate of small targets at high altitudes of UAVs.

[0063] Specifically, the process of constructing a non-rigid target detector and adding an offset prediction branch to adaptively match the irregular shape of the construction target is as follows:

[0064] The reconstructed construction image is input into the non-rigid target detector; the non-rigid target detector extracts the depth feature map of the reconstructed construction image through a backbone network; the offset prediction branch learns and predicts the deformation offset of the target region based on the depth feature map, and generates an offset field.

[0065] The non-rigid target detector uses the offset field to perform adaptive feature sampling on the depth feature map to obtain alignment features of the irregular shape and posture of the construction target; based on the alignment features, it outputs a target instance segmentation mask and a posture key point heatmap through two parallel prediction heads.

[0066] The non-rigid target detector extracts the depth feature map of the reconstructed construction image through a backbone network by using a deep convolutional neural network as a feature extractor. The input multi-channel reconstructed construction image is subjected to layer-by-layer convolution and downsampling processing to extract a feature tensor containing high-level semantic information and abstract texture features. The resolution of this feature tensor is lower than that of the input image, but the number of channels is increased, which can characterize the structural information in the image.

[0067] The offset prediction branch learns and predicts the deformation offset of the target region based on the depth feature map. The offset field is generated by adding an additional convolutional layer in parallel at the output of the backbone network as an offset generator. This generator does not perform target classification, but is specifically used to regress and calculate the spatial position deviation of the sampling points of the convolution kernel. For each sampling point in the convolution kernel, this branch outputs the corresponding horizontal and vertical coordinate offset values. The coordinate offset values ​​of all sampling points together constitute the offset field corresponding to the size of the input feature map.

[0068] To perform deformable convolution, a priori mask of the human body contour is first introduced as a constraint, and the initial coordinate offset value is calculated. The priori mask of the human body contour is generated based on temporal motion information: inter-frame difference or background modeling is performed using construction images from adjacent frames to quickly extract the moving foreground region, and this foreground region is used as the priori mask for constrained sampling points. Before superimposing the offset value onto the sampling coordinates, boundary out-of-bounds judgment logic is executed: it is determined whether the coordinates of the superimposed sampling points fall within the effective human body area defined by the priori mask. For outlier sampling points that fall outside the area, backtracking correction is performed along the opposite direction of their offset vector to force them to be mapped back to the human body edge boundary. This ensures that the corrected convolution kernel sampling points can be closely attached to the effective pixel area of ​​the non-rigid target. A bilinear interpolation algorithm is used to calculate the pixel value at the non-integer coordinates, thereby generating alignment features that can be strictly confined within the human body topology.

[0069] This invention addresses the problem of traditional deformable convolution sampling points drifting to the background region by introducing prior mask constraints and boundary backtracking correction mechanisms. It forces the convolution kernel sampling points to closely fit the effective pixel area of ​​the human body, thereby eliminating background noise interference and further enhancing the extraction accuracy of non-rigid contour edge information of the human body by mapping outliers to the boundary.

[0070] The alignment features are simultaneously input into two structurally independent sub-networks. One sub-network is a fully convolutional branch, which restores spatial resolution through upsampling and performs binary classification on each pixel using the Sigmoid activation function, outputting a binary mask representing the target foreground region. The other sub-network is a keypoint regression branch, which predicts the coordinate positions of human skeletal keypoints in the image plane through convolutional layers, and generates a probability heatmap conforming to a Gaussian distribution centered on these coordinates. The peak position of this heatmap represents the predicted position of the keypoint.

[0071] By introducing offset prediction branches and deformable convolution mechanisms in the feature extraction stage, the problem that traditional rigid convolution kernels cannot effectively adapt to the diverse physical forms of people on construction sites is fundamentally solved. By allowing the convolution kernels to deform according to the shape of the target, it is ensured that the feature extraction process always focuses on effective areas such as the human torso and limbs, rather than collecting background noise, thereby greatly improving the segmentation edge fitting and key point positioning accuracy of non-rigid targets in complex construction environments.

[0072] The specific process of outputting a target instance segmentation mask and a pose key point heatmap through two parallel prediction heads is as follows:

[0073] The prediction head includes a first prediction head and a second prediction head; the first prediction head generates a preliminary target instance segmentation mask based on the alignment features, and uses the preliminary target instance segmentation mask to spatially weight the alignment features to generate focusing features; the second prediction head performs localization regression based on the focusing features to generate the pose key point heatmap, and extracts the structural features of the pose key point heatmap and feeds them back to the first prediction head; the first prediction head uses the structural features to perform edge correction on the preliminary target instance segmentation mask and outputs the target instance segmentation mask;

[0074] The first prediction head is a fully convolutional neural network branch. This branch receives the alignment features as input, extracts semantic information through a series of convolutional layers, and performs upsampling operations to restore the spatial resolution of the image. The probability value of each pixel belonging to the construction target category is calculated using the Sigmoid activation function. A rough preliminary target instance segmentation mask is generated by setting a probability threshold to indicate the approximate location and outline of the target.

[0075] The initial target instance segmentation mask is used as a spatial attention weight map. After normalizing its numerical range, it is multiplied by the alignment feature in a channel-by-channel pixel-level operation. The physical meaning of this operation is to suppress the numerical response of the alignment feature corresponding to the background region, while enhancing the feature intensity corresponding to the human target region, thereby obtaining a focusing feature that removes background noise interference and retains only the effective information of the target.

[0076] A keypoint estimation network branch is used as the second prediction head. This branch specifically receives the cleaned focused features and uses a regression network to roughly predict the integer grid coordinates of each human skeleton keypoint on the image. It further predicts the small offset of the grid coordinates relative to the real sub-pixel position. When generating the Gaussian distribution response, the offset is introduced to perform fine translation calibration of the center point of the Gaussian kernel at the sub-pixel level to eliminate the spatial quantization error caused by feature map downsampling. Finally, a multi-channel probability density map with the center position strictly aligned with the real pixel is output, which is the pose keypoint heatmap.

[0077] This embodiment employs an innovative sub-pixel level calibration mechanism, which corrects the center position of the Gaussian distribution response by predicting a tiny offset relative to the integer grid, effectively overcoming the spatial quantization error caused by feature map downsampling in deep learning networks.

[0078] A feature encoding conversion module is set up. The output pose keypoint heatmap is input into this module. Through convolution operation, the spatial distribution information of the two-dimensional heatmap is remapped back to the high-dimensional feature channel space, generating a structural feature tensor containing the topological structure of the human skeleton and the limb connection relationship. This tensor is then transmitted back to the end-processing stage of the first predicted head. The feature encoding conversion module contains a three-layer convolutional neural network. The first layer uses... Convolution kernel. The second layer uses... The convolution kernel and the third layer output a structural feature tensor with the same number of channels as the first prediction head feature layer. This structural feature tensor is then concatenated with the feature layer before the first prediction head generates the initial mask in the channel dimension.

[0079] The structural features fed back are spliced ​​or fused with the feature layer before generating the initial mask. The rigid structural constraints of the human skeleton are used to correct the edges of the flexible segmentation mask. In particular, for areas where limbs cross, overlap, or are blurred, the pixel ownership is re-determined based on the bone direction, thereby removing the overflowing background pixels and filling the missing target pixels, and outputting the target instance segmentation mask.

[0080] By constructing a two-way feature interaction mechanism of segmentation-guided localization and posture feedback segmentation, and using the preliminary segmentation mask as spatial attention weight, noise interference from complex backgrounds at construction sites is effectively suppressed, and the signal-to-noise ratio of key point localization is significantly improved. At the same time, the topological features of the human skeleton are fed back to the segmented head, and the rigid constraints of the skeleton are used to correct the edges of the flexible segmentation mask. This effectively solves the technical problem that the segmentation contour is prone to overflow or missing in scenarios with limb intersection, occlusion, or motion blur, and achieves high-precision morphological reconstruction and edge alignment of non-rigid construction targets.

[0081] The specific process for identifying dangerous area approach violations based on the target instance segmentation mask and posture key point heatmap includes: obtaining the dangerous area vector boundary of the construction site; performing coordinate analysis on the posture key point heatmap to extract the coordinates of foot key points representing the human standing position, and calculating the shortest Euclidean distance from the foot key point coordinates to the dangerous area vector boundary; calculating the overlapping pixel area of ​​the region enclosed by the target instance segmentation mask and the dangerous area vector boundary, and obtaining the body intrusion ratio; constructing a joint discrimination logic including the shortest Euclidean distance and the body intrusion ratio, and determining that there is a dangerous area approach violation when the shortest Euclidean distance is less than a preset safety threshold (set to 1 meter in this embodiment) or the body intrusion ratio exceeds a preset warning threshold (set to 10% in this embodiment);

[0082] In the pixel coordinate system of the construction image, the outlines of dangerous areas such as foundation pits, edges, and high-voltage equipment are pre-calibrated manually or automatically extracted through semantic segmentation. These outlines are then stored as polygon vector data composed of a series of ordered vertex coordinates, or converted into a binary dangerous area mask map with the same size as the construction image. In this mask map, the pixel value of dangerous areas is 1, and the pixel value of non-dangerous areas is 0.

[0083] The process of analyzing the coordinates of the posture key point heatmap to extract the coordinates of the foot key points representing the human standing position specifically involves traversing the channels corresponding to the left and right ankles in the posture key point heatmap, finding the pixel position with the largest response value as the positioning point of the left and right feet. If both feet are detected simultaneously, the geometric center coordinates of the line connecting the two points are calculated as the coordinates of the foot key points. If only one foot is detected, the coordinates of that foot are directly taken as the coordinates of the foot key points. These coordinates physically represent the point of force and standing position of the construction worker on the ground. Using the distance formula from a point to a line segment, the vertical distance or endpoint distance from the foot key point coordinates to each edge line segment in the polygon vector data is calculated. All results are traversed, and the minimum value is selected as the shortest Euclidean distance. This distance value intuitively reflects the spatial distance between the person's standing position and the danger edge.

[0084] The target instance segmentation mask and the binarized danger zone mask are logically ANDed. The total number of pixels with a value of 1 in the result is counted to obtain the overlapping pixel area. Then, the overlapping pixel area is divided by the total pixel area of ​​the target instance segmentation mask itself. The quotient is the body intrusion ratio, which represents how much of the human body is suspended in the air or inside the danger zone. A distance threshold (e.g., 50 pixels, corresponding to a real distance of 1 meter) and a ratio threshold (e.g., 10%) are set. The logical judgment adopts an OR relationship: as long as a person is detected to be standing too close (distance less than the threshold) or standing far but protruding too much (e.g., leaning into a deep well, intrusion ratio greater than the threshold), a violation alarm signal is triggered. This dual discrimination mechanism effectively prevents missed detections caused by relying solely on distance judgment (e.g., upper body protruding into the danger zone) or false alarms caused solely on overlap rate judgment.

[0085] By constructing a dual discrimination model combining physical distance and spatial overlap, the limitations of a single judgment dimension are overcome. Distance is calculated using key foot points, which aligns with the principle in physics that the standing point determines position, accurately identifying edge risks. The intrusion ratio is calculated using mask overlap, compensating for blind spots in identification where, in special postures such as leaning forward or operating in mid-air, the feet may be in a safe zone but the body may be in a danger zone. This aligns with real-world safety management standards at construction sites and significantly improves the accuracy of identifying complex violations.

[0086] The specific process for identifying violations related to safety protective clothing based on correlation graphs includes:

[0087] The pose key point heatmap is multiplied pixel-by-pixel with the corresponding target instance segmentation mask to generate a correlation map. The high-response regions of the correlation map are used as spatial attention weights to weighted aggregate image features to generate local attribute feature vectors. The semantic feature distance between the local attribute feature vectors and the preset compliance prototype vectors is calculated, and the geometric contour deviation of the target part is calculated based on the gradient distribution of the correlation map. If the weighted sum of the semantic feature distance and the geometric contour deviation exceeds a preset threshold, a violation of safety protection wear category is determined.

[0088] The specific process for generating the correlation map is as follows:

[0089] The pose key point heatmap and the target instance segmentation mask are unified to the same spatial scale; the target instance segmentation mask is used to perform a spatial masking operation on the pose key point heatmap to eliminate background interference signals in non-target areas; nonlinear mapping is performed on the masked image, and morphological filtering is used to remove isolated noise areas to generate the correlation map.

[0090] Specifically, the difference in pixel resolution between the pose key point heatmap and the target instance segmentation mask is detected. Typically, the heatmap is a low-resolution feature output by the backbone network downsampling, while the mask is a high-resolution binary image recovered by upsampling. A bilinear interpolation algorithm is used to enlarge the width and height of the pose key point heatmap to the same pixel size as the target instance segmentation mask, thereby constructing a pixel-by-pixel spatial coordinate system between the two.

[0091] The target instance segmentation mask is used as a binary gating filter, and a Hadamard product operation is performed on it with the magnified pose key point heat map, i.e., multiplying pixel by pixel. For background areas with a mask value of 0, the corresponding heat map response value is forced to be set to zero, thereby completely filtering out false high-frequency responses caused by background texture, shadow or lighting noise outside the human body contour, ensuring that subsequent processing only focuses on the effective human body area.

[0092] The nonlinear mapping of the masked image specifically involves first normalizing the pixel values ​​of the masked heatmap to the range of zero to one, and then applying the Sigmoid activation function for pixel-level recalibration. This aims to stretch the dynamic range of pixel values, suppress the low-confidence blur response range (i.e., suppress weak signals), and at the same time nonlinearly enhance the contrast of the peak region of high-confidence key points, making key parts (such as the head and shoulders) in the heatmap more significant and concentrated in terms of value.

[0093] Define a rectangle of a preset size, perform morphological opening operation on the image after nonlinear mapping, that is, first perform erosion operation to peel off and eliminate isolated noise points or discrete patches with an area smaller than the structuring element, then perform dilation operation to restore the morphological integrity of the main high-response areas, and finally output the correlation map with a clean background, connected target areas and smooth edges.

[0094] By unifying the spatial scale and using forced masking based on segmentation masks, hard isolation between the background and the target is achieved in physical space, effectively solving the problem of false activation caused by background noise (such as construction site debris similar to skin color) in traditional heatmap analysis. By combining nonlinear mapping and morphological opening operation techniques, the signal-to-noise ratio of the features is further enhanced at the signal level, and random discrete noise points in the image are removed, so that the generated correlation map can represent the spatial topological morphology of specific body parts of construction workers with extremely high accuracy.

[0095] The correlation map is downsampled to the same size as the deep feature map output by the backbone network and used as a spatial attention mask. Global weighted average pooling is performed on each channel of the deep feature map, that is, the feature values ​​at the corresponding positions of the feature map are weighted and summed using the pixel values ​​of the correlation map, thereby compressing the two-dimensional image features into a one-dimensional feature vector. This vector only encodes the visual attributes of specific parts of the human body (such as the head region) and shields the interference of the background and other body parts.

[0096] Collect a large number of positive samples of people wearing compliant equipment (such as wearing safety helmets and reflective clothing), extract their feature vectors and calculate cluster centers to obtain the preset compliance prototype vector; in the recognition stage, calculate the cosine distance between the currently extracted local attribute feature vector and the preset compliance prototype vector. The larger the distance value, the greater the semantic difference between the current target's feature attributes (such as color, texture, material) and the standard compliance state.

[0097] The gradient direction histogram of edge pixels of high response region in the correlation map is calculated to construct the shape descriptor of the current target. It is then compared with the preset standard compliant shape descriptor (e.g., the semi-circular outer contour feature of a standard safety helmet). The difference between the two is calculated as the geometric contour deviation, which reflects whether the target part has the unique geometric features of protective equipment in terms of physical shape.

[0098] If the weighted sum of the semantic feature distance and the geometric contour deviation exceeds a preset threshold, a violation of the safety protection wear category is determined. This involves setting two weight coefficients to correspond to the importance of semantic features and geometric shape, respectively, and calculating the weighted sum of the two distance values. If the sum is greater than a preset determination threshold (set to 0.6 in this embodiment), it indicates that the target does not meet compliance requirements in terms of material properties and does not have protective features in terms of geometric shape, thereby outputting a violation alarm signal.

[0099] By constructing a dual-discrimination model based on semantic feature measurement and geometric topological constraints, the limitations of traditional methods that rely solely on color thresholds or simple classification labels are overcome. By using correlation maps as a spatial attention mechanism, noise interference from complex construction backgrounds is effectively removed, ensuring the purity of local attribute feature extraction. By introducing prototype vector comparison and geometric contour deviation calculation, not only is high robustness to changes in lighting and texture differences achieved, but also the ability to accurately distinguish easily confused scenarios such as holding a safety helmet versus wearing a safety helmet at the physical morphology level is significantly improved, enhancing the fine-grainedness and accuracy of safety violation identification.

[0100] This application effectively solves the technical challenges of easily losing small target features and truncating grid boundary information under the high-altitude top-down view of UAVs by adopting α data augmentation and dual-path translational misalignment and recombination mechanism; combined with a non-rigid target detector that includes an offset prediction branch, it achieves adaptive and accurate perception of irregular shapes such as construction workers bending over and climbing; furthermore, it constructs a joint discrimination logic by using the correlation graph generated by attitude key points and segmentation mask, which realizes feature decoupling between target and background in physical space, significantly improving the recognition accuracy and robustness of complex violations such as intrusion into dangerous areas and wearing protective equipment.

[0101] Example 2

[0102] This embodiment demonstrates the application of the intelligent identification method for construction safety violations based on UAV imagery proposed in this invention to a specific scenario during the construction of a deep foundation pit. It mainly targets the edge work area at the edge of the foundation pit, identifying and alarming the dangerous intrusion behavior of construction personnel (i.e., getting too close to the edge of the foundation pit or climbing over the fence).

[0103] An industrial drone equipped with a high-resolution gimbal camera was selected and set to fly at an altitude of 45 meters above the pit working surface to acquire original RGB construction images with a resolution of 3840×2160. To adapt to the input requirements of the deep neural network model (set a uniform size of 1024×1024), the width and height scaling ratios were calculated, and the smaller value was selected as the image magnification factor α to scale the image proportionally to obtain an intermediate image, which was then placed in the center of the canvas.

[0104] For the blank edge filling areas on the top and bottom sides, the original high-resolution image is returned, and N source data blocks are randomly generated rectangular selection boxes. Adaptation is carried out by combining loop stitching and edge clipping: the clipped source data blocks are used as basic texture units and continuously laid in the edge filling area in the extension direction. When the laying range exceeds the boundary, the excess pixels are directly clipped, which ensures that the filling area has a real construction texture and does not change the original geometric ratio of the pixels, avoiding texture deformation caused by forced stretching.

[0105] To prevent the loss of minute features of workers at the edge of the excavation pit from an aerial view during downsampling, standard construction images are processed in dual-path mode:

[0106] First Recombination Path: Perform spatial-to-depth transformation according to the set block size to generate the first recombined feature map;

[0107] The second reconstruction path: First, perform a fixed spatial translation on the image, with an offset of half the block size (i.e., 1 pixel). Then, perform a spatial-to-depth transformation to generate a second reconstruction feature map. Finally, concatenate the two feature maps in the channel dimension to generate a reconstructed construction image, ensuring that the features of the worker are completely preserved regardless of whether the worker is located at the center or the edge of the grid.

[0108] To address the irregular postures that workers might exhibit near the foundation pit, such as leaning forward to observe or bending over, a non-rigid target detector is constructed. The offset prediction branch generates an offset field based on the depth feature map to guide the convolutional kernel in sampling.

[0109] During this process, the boundary crossing judgment logic is executed: a human body contour prior mask is introduced. If the offset sampling point is detected to fall into the background area (such as the soil in the foundation pit), the backtracking correction is performed along the opposite direction of its offset vector to force it to be mapped back to the human body edge boundary. Then, bilinear interpolation is used to calculate the pixel value to generate alignment features that closely fit the irregular posture of the worker and eliminate background noise interference.

[0110] Based on alignment features, two parallel prediction heads work together:

[0111] The first predictive head generates a preliminary target instance segmentation mask and uses it as a spatial weight to generate focusing features; the second predictive head receives the focusing features and uses a regression network to predict the coordinates of key points of the skeleton and the small offset relative to the sub-pixel position. When generating the pose key point heatmap, the offset is introduced to perform sub-pixel level fine translation calibration on the center of the Gaussian kernel to eliminate quantization error and ensure that the center of the heatmap is strictly aligned with the actual joint position of the worker.

[0112] Meanwhile, the skeletal structure features extracted by the second head are fed back to the first head, and the rigid constraints of the skeleton are used to correct the edge of the segmentation mask, thus solving the problem of contour adhesion when the worker's limbs overlap with the pit guardrail.

[0113] The posture key point heatmap is analyzed to extract the coordinates of the foot key points representing the worker's standing position. The shortest Euclidean distance from the foot to the vector boundary of the pit edge is calculated. The overlapping pixel area of ​​the area enclosed by the target instance segmentation mask and the vector boundary of the danger zone is calculated to obtain the body intrusion ratio. A joint discrimination logic is constructed. When the shortest Euclidean distance of a worker is detected to be less than the preset safety threshold (e.g., 1 meter), or the body intrusion ratio exceeds the preset warning threshold (e.g., 15%), it is determined that the target has a danger zone approach violation and is reported.

[0114] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent identification of construction safety violations based on UAV imagery, characterized in that: The construction images to be identified are collected by drones, and the construction images are magnified according to the image magnification factor α to generate standard construction images. The standard construction images are then subjected to spatial to depth transformation by standard grid and translational misalignment grid, and concatenated and merged in the channel dimension to generate reconstructed construction images. A non-rigid target detector is constructed and an offset prediction branch is added. Based on the reconstructed construction image, the irregular shape of the construction target is adaptively matched to generate a target instance segmentation mask and a posture key point heatmap. The target instance segmentation mask is a binary region map of the target contour, and the posture key point heatmap is a probability distribution map representing the position of key points. Based on the target instance segmentation mask and the posture key point heatmap, the violation of proximity to dangerous areas is identified. The attitude key point heatmap is multiplied pixel by pixel with the corresponding target instance segmentation mask to generate a correlation map. Based on the correlation map, violations related to safety protective clothing are identified; and the identified construction safety violations are reported.

2. The intelligent identification method for construction safety violations based on UAV imagery according to claim 1, characterized in that, The specific process of magnifying the construction image according to the image magnification factor α to generate a standard construction image is as follows: The standard construction image is pre-defined to a uniform size; the original size of the construction image is obtained, the width scaling ratio and the height scaling ratio are calculated, the minimum value of the two is selected as the image magnification factor α, and the construction image is scaled proportionally according to the image magnification factor α to obtain an intermediate image; In the space of the uniform size, a central main region and an edge filling region are defined, and the intermediate image is placed in the central main region. Returning to the unscaled original construction image, N source data blocks are randomly cropped from the original construction image. The source data blocks are interpolated and scaled, and the size is adapted to the edge filling region by combining cyclic stitching and edge cropping. The scaled source data blocks are used to fill the edge filling region to generate the standard construction image.

3. The intelligent identification method for construction safety violations based on UAV imagery according to claim 1, characterized in that, The specific process for generating the reconstructed construction images is as follows: Set the block size and set at least two parallel reassembly paths, including a first reassembly path and a second reassembly path; The first reconstruction path performs a space-to-depth transformation on the standard construction image to generate a first reconstruction feature map; The second reconstruction path performs spatial translation on the standard construction image, and then performs the spatial-to-depth transformation on the translated image to generate a second reconstructed feature map; The first reconstructed feature map and the second reconstructed feature map are concatenated along the channel dimension to generate a reconstructed construction image.

4. The intelligent identification method for construction safety violations based on UAV imagery according to claim 1, characterized in that, The specific process of constructing a non-rigid target detector and adding an offset prediction branch to adaptively match the irregular shape of the construction target is as follows: The reconstructed construction image is input into the non-rigid target detector; the non-rigid target detector extracts the depth feature map of the reconstructed construction image through a backbone network; the offset prediction branch learns and predicts the deformation offset of the target region based on the depth feature map, and generates an offset field. The non-rigid target detector uses the offset field to perform adaptive feature sampling on the depth feature map to obtain alignment features for aligning the irregular shape and posture of the construction target; Based on the alignment features, target instance segmentation masks and pose key point heatmaps are output through two parallel prediction heads.

5. The intelligent identification method for construction safety violations based on UAV imagery according to claim 4, characterized in that, The specific process of outputting a target instance segmentation mask and a pose key point heatmap through two parallel prediction heads is as follows: The prediction head includes a first prediction head and a second prediction head; the first prediction head generates a preliminary target instance segmentation mask based on the alignment features, and uses the preliminary target instance segmentation mask to spatially weight the alignment features to generate focusing features; The second prediction head performs localization regression based on the focusing features to generate the attitude key point heatmap, and extracts the structural features of the attitude key point heatmap and feeds them back to the first prediction head; the first prediction head uses the structural features to perform edge correction on the preliminary target instance segmentation mask and outputs the target instance segmentation mask.

6. The intelligent identification method for construction safety violations based on UAV imagery according to claim 1, characterized in that, The specific process of identifying proximity violations in dangerous areas based on the target instance segmentation mask and pose key point heatmap includes: Obtain the vector boundary of the hazardous area at the construction site; perform coordinate analysis on the posture key point heatmap to extract the coordinates of the foot key points representing the human standing position, and calculate the shortest Euclidean distance from the foot key point coordinates to the vector boundary of the hazardous area; calculate the overlapping pixel area of ​​the region enclosed by the target instance segmentation mask and the vector boundary of the hazardous area, and obtain the body intrusion ratio; construct a joint discrimination logic including the shortest Euclidean distance and the body intrusion ratio, and determine that there is a violation of approaching a hazardous area when the shortest Euclidean distance is less than a preset safety threshold or the body intrusion ratio exceeds a preset warning threshold.

7. The intelligent identification method for construction safety violations based on UAV imagery according to claim 1, characterized in that, The specific process for identifying violations related to safety protective clothing based on correlation graphs includes: The pose key point heatmap is multiplied pixel-by-pixel with the corresponding target instance segmentation mask to generate a correlation map. The high-response regions of the correlation map are used as spatial attention weights to weighted aggregate image features to generate local attribute feature vectors. The semantic feature distance between the local attribute feature vectors and the preset compliance prototype vectors is calculated, and the geometric contour deviation of the target part is calculated based on the gradient distribution of the correlation map. If the weighted sum of the semantic feature distance and the geometric contour deviation exceeds a preset threshold, a violation of safety protection wear category is determined.

8. The intelligent identification method for construction safety violations based on UAV imagery according to claim 7, characterized in that, The specific process for generating the correlation map is as follows: The pose key point heatmap and the target instance segmentation mask are unified to the same spatial scale; the target instance segmentation mask is used to perform a spatial masking operation on the pose key point heatmap to eliminate background interference signals in non-target areas; nonlinear mapping is performed on the masked image, and morphological filtering is used to remove isolated noise areas to generate the correlation map.

Images