Construction site safety monitoring target detection method based on multi-scale feature fusion

Abstract

The invention requests to protect a construction site safety monitoring target detection method based on multi-scale feature fusion. The method comprises the following steps: preprocessing a construction site monitoring image to generate a training and test set; constructing a target detection model containing a feature extraction backbone network, a feature fusion neck network and an RTDETRDecoder detection head; inputting data to train and test the model. The method has the advantages that (1) multi-cardinal SENetV2 models channel dependence from multiple angles, and feature expression is enhanced; (2) Zoomcat is fused with multi-scale features, semantics and details are considered, and ScalSeq is used for modeling a scale sequence relation through 3D convolution, so that the small target detection precision is improved; (3) the lightweight design realizes 80 FPS real-time reasoning, the mAP is improved by 4.1% compared with that of YOLOv5, and the real-time and accurate requirements of construction site monitoring are met; and (4) edge feature enhancement and de-noising training improve the robustness of the complex environment.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method for target detection in construction site safety monitoring based on multi-scale feature fusion. Background Technology

[0002] With the rapid development of the construction industry, construction site safety management has become increasingly important. Traditional construction site safety monitoring mainly relies on manual inspections, which suffers from low efficiency, poor real-time performance, and easy omissions. Intelligent monitoring systems based on computer vision can monitor the safety status of construction sites in real time, automatically identify safety violations, and provide effective support for construction site safety management.

[0003] Existing target detection methods in construction site safety monitoring applications mainly suffer from the following problems: insufficient small target detection capability. In construction site scenarios, targets such as safety helmets and personnel are often far from the camera and appear as small targets in the image. Traditional target detection methods have low detection accuracy for small targets; insufficient multi-scale feature fusion. Traditional methods typically use simple upsampling or downsampling when fusing features at different scales, failing to fully utilize the complementarity of multi-scale information. Although multi-scale feature fusion is used, the fusion method is relatively simple and it is difficult to effectively extract cross-scale contextual information; low efficiency of attention mechanisms. Traditional SE attention mechanisms use a single fully connected layer for channel compression, which has limited expressive power and is difficult to capture complex channel dependencies; high real-time performance requirements. Construction site safety monitoring requires real-time processing of video streams, which places high demands on detection speed. How to improve inference speed while ensuring detection accuracy remains a problem to be solved; poor adaptability to complex scenes. Construction site environments are complex, with large changes in lighting, and problems such as occlusion and blurring exist. Existing methods lack robustness.

[0004] A search revealed application publication number CN113313118A, entitled "A Method for Target Detection in Construction Site Safety Monitoring Based on Multi-Scale Feature Fusion," specifically an adaptive variable-scale target detection method based on multi-scale feature fusion. This invention belongs to the field of target detection technology. It addresses the problem that when the target is a small target, its small size on the feature map makes it susceptible to being dominated by larger targets during the learning process, leading to poor detection results. The method includes: a variable-scale image generator generating target data of different sizes from the original data; then feeding the target data to a feature extraction network; the feature extraction network extracts features; and then feeding the extracted features to a multi-scale feature generator. The multi-scale feature generator produces feature maps at three scales: a small feature map, a medium feature map, and a large feature map. These three feature maps are then fed into a multi-scale prediction box, which generates prediction boxes for each of the three scale feature maps. The content within the prediction box represents the target detection result. This invention is used for target detection, and is particularly suitable for the detection of small targets.

[0005] Patent CN 113313118 A improves small target detection capabilities through a multi-scale feature fusion mechanism, but it neglects key issues such as edge feature degradation in complex scenes, single channel dependency modeling, and lack of scale sequence correlation. This results in insufficient capture of multi-dimensional target attributes, leading to lower detection accuracy in practical scenarios such as construction site safety monitoring. This invention improves feature representation capabilities by introducing edge feature enhancement and a multi-cardinality attention mechanism. The edge feature enhancement module explicitly strengthens the contour information of small targets by fusing edge detection results with the original image. The multi-cardinality attention mechanism uses parallel fully connected layers to model channel dependencies from multiple angles, enhancing the expressive power of key features.

[0006] Furthermore, this invention innovatively designs an adaptive feature fusion module and a 3D sequence modeling module. The adaptive fusion module unifies multi-scale features through intelligent pooling and interpolation strategies, taking into account both semantic information and detailed features; the 3D sequence modeling module converts multi-scale features into three-dimensional tensors, effectively capturing the correlation between scales. By systematically optimizing the edge details, channel weights, and scale relationships of target features, the detection accuracy is significantly improved while maintaining real-time performance, making it particularly suitable for complex application scenarios such as construction site safety monitoring.

[0007] Therefore, there is a need for a target detection method that can effectively detect safety targets on construction sites, improve detection accuracy, and maintain real-time performance. Summary of the Invention

[0008] The technical problem to be solved by this invention is: how to design a multi-scale feature fusion mechanism to effectively improve the accuracy of target detection in construction site safety monitoring, especially the detection accuracy of small targets, while maintaining real-time performance.

[0009] To address the aforementioned technical problems, this invention provides a target detection method for construction site safety monitoring based on multi-scale feature fusion. The core innovation of this invention lies in its design of a multi-level feature enhancement and fusion mechanism to address the difficulty of detecting small targets in construction site safety monitoring scenarios. This mechanism includes a multi-cardinality attention mechanism, adaptive multi-scale feature fusion, and 3D sequence feature modeling, improving detection accuracy while maintaining real-time performance. A target detection method for construction site safety monitoring based on multi-scale feature fusion is proposed.

[0010] The technical solution of the present invention is as follows:

[0011] A target detection method for construction site safety monitoring based on multi-scale feature fusion includes the following steps:

[0012] S1, preprocess the collected construction site monitoring images to generate training and test datasets;

[0013] S2, Construct an object detection model. The input to the object detection model is the training data and the test data, and the output of the object detection model is the object recognition result. The training data is the data in the training dataset, and the test data is the data in the test dataset. The object detection model includes: a feature extraction backbone network, a feature fusion neck network, and a detection head network. The data output end of the feature extraction backbone network is connected to the data input end of the feature fusion neck network, and the data output end of the feature fusion neck network is connected to the data input end of the detection head network.

[0014] The feature extraction backbone network adopts the ResNet18 architecture and embeds an SENetV2 (multi-cardinality channel attention mechanism network) attention module in each basic block. The SENetV2 attention module adopts a multi-cardinality channel attention mechanism to adaptively adjust the weights of the input features.

[0015] The feature fusion neck network includes: an AIFI (Internal Scale Feature Interaction) module, used to perform internal scale feature interaction on the highest-level features output by the backbone network, flattening the feature map into a sequence and processing it through a Transformer encoder; a Zoom_cat (Adaptive Multi-Scale Feature Fusion) module, used to receive feature maps of three different scales, downsampling the largest-scale features using adaptive max pooling and average pooling, upsampling the smallest-scale features using nearest-neighbor interpolation, unifying them to a medium scale, and then concatenating them along the channel dimension; and RepC3 (a lightweight network based on reparameterized convolutions). The first module, ScalSeq, is used to construct a C3 bottleneck structure using reparameterized convolutions, and then performs lightweight processing on the fused features. The second module, ScalSeq (scale sequence feature processing), receives three feature maps P3, P4, and P5 at different scales. It unifies the number of channels using 1×1 convolutions, upsamples P4 and P5 to the P3 scale, converts the three feature maps into 3D tensors, processes them using 3D convolutions, batch normalization, and the LeakyReLU activation function, and finally outputs them after 3D max pooling and dimensionality compression. The output of the ScalSeq module is then added element-wise to the P3 features processed by RepC3 using the Add module.

[0016] The detection head network uses RTDETRDecoder (decoder) to receive fused multi-scale features and generate target bounding boxes and class predictions through deformable Transformer decoder;

[0017] Training is performed using the RTDETRDetectionLoss (RTDETR loss function), which includes classification loss, bounding box regression loss, and denoising training loss.

[0018] S3, Input the training data to train the target detection model;

[0019] S4, Input the test data to test the target detection model;

[0020] Furthermore, S1 preprocesses the acquired construction site monitoring images to generate training and testing datasets, specifically including:

[0021] S1-1, collects construction site monitoring images, including safety helmet wearing detection, personnel behavior detection, dangerous area intrusion detection, and equipment status detection scenarios;

[0022] S1-2, Use the annotation tool to annotate the target location information in each image to obtain the annotation file. The annotation categories include: safety helmet, person not wearing a safety helmet, dangerous area, safety equipment, and dangerous behavior;

[0023] S1-3, Convert the format of the annotation file to obtain the dataset;

[0024] S1-4 divides the dataset into a training dataset and a test dataset, with a ratio of 8:2 between the training dataset and the test dataset.

[0025] Furthermore, S1 also includes:

[0026] The image is first processed using an image processing algorithm to obtain a grayscale image containing edge features. The image processing algorithm includes one or any combination of grayscale conversion, binarization, erosion, dilation, denoising, and edge detection.

[0027] The grayscale image containing edge features is then fused with the original three-channel image to obtain a four-channel image containing edge features, which is used as input to the target detection model.

[0028] Furthermore, the specific implementation of the SENetV2 attention module includes:

[0029] S2.1: Perform global average pooling on the input feature map to obtain channel descriptors;

[0030] S2.2: Input the channel descriptors into four parallel fully connected layers respectively. Each fully connected layer compresses the number of channels to 1 / reduction of the original number of channels, where reduction=16;

[0031] S2.3: The outputs of the four fully connected layers are concatenated along the channel dimension to obtain multi-cardinality fusion features;

[0032] S2.4: The fused features are restored to the original number of channels through a fully connected layer, and channel weights are generated through the Sigmoid activation function;

[0033] S2.5: Multiply the channel weights element-wise with the original feature map to obtain the weighted feature map.

[0034] Furthermore, the specific implementation of the Zoom_cat module includes:

[0035] S3.1: Receive three input feature maps l, m, and s, representing large, medium, and small scales, respectively;

[0036] S3.2: Downsample the large-scale feature l to a medium-scale m using adaptive max pooling and adaptive average pooling, and then add the two pooling results together;

[0037] S3.3: Upsample the small-scale feature s to a medium-scale m using nearest neighbor interpolation;

[0038] S3.4: Concatenate the processed l, m, and s feature maps along the channel dimension to output the fused feature map.

[0039] Furthermore, the specific implementation of the ScalSeq module includes:

[0040] S4.1: Receive three feature maps P3, P4, and P5 of different scales, and unify the number of channels to a single channel by performing 1×1 convolution on each map.

[0041] S4.2: Upsample P4 and P5 to the spatial dimensions of P3 using nearest neighbor interpolation;

[0042] S4.3: Add a dimension to the third dimension of P3, P4_2, and P5_2 respectively, and convert them into 3D tensors;

[0043] S4.4: Concatenate three 3D tensors along the third dimension to form a combined 3D tensor;

[0044] S4.5: Perform 3D convolution (kernel_size=(1,1,1)), batch normalization, and LeakyReLU activation on the combined 3D tensor;

[0045] S4.6: Process the data using 3D max pooling (kernel_size=(3,1,1)) and compress the third dimension to output the final feature map.

[0046] Furthermore, the RTDETRDetectionLoss loss function includes:

[0047] S6.1: Classification Loss: The difference between the predicted class and the true class is calculated using Varifocal Loss (zoom focus loss function). The specific calculation formula is as follows:

[0048] ;

[0049] in To predict class probabilities, For real category labels, The number of samples;

[0050] S6.2: Bounding Box Regression Loss: GIoU Loss is used to calculate the difference between the predicted and true bounding boxes. The specific calculation formula is as follows:

[0051] ;

[0052] in To predict the bounding box, For the true bounding box, The calculation formula is:

[0053] ;

[0054] in For intersection, union, and comparison, Let be the minimum bounding rectangle of the predicted bounding box and the ground truth bounding box. For the prediction box, For true bounding boxes;

[0055] S6.3: Denoising Training Loss: During the training phase, the query with added noise is denoised and trained, and the loss between the denoised prediction and the true value is calculated.

[0056] The total loss is:

[0057] ;

[0058] in This represents the training loss during denoising.

[0059] Furthermore, the target detection model also includes:

[0060] Replacing some standard convolution operations with depthwise separable convolution, which includes channel-wise convolution and pointwise convolution;

[0061] The term "channel-wise convolution" refers to the fact that each convolutional kernel is responsible for only one single channel, mainly for filtering.

[0062] The pointwise convolution is a weighted combination of the feature maps of the previous layer in the depth direction to generate new feature maps;

[0063] The specific calculation method for the depthwise separable convolution is as follows:

[0064] ;

[0065] in:

[0066] This indicates the computational cost of depthwise separable convolution. Indicates the length of the feature map; Indicates the width of the feature map;

[0067] Indicates the number of channels in the feature map; Indicates the length of the convolution kernel; Indicates the width of the convolution kernel; This indicates the number of convolution kernels.

[0068] A construction site safety monitoring target detection device based on multi-scale feature fusion, comprising:

[0069] A processor for executing any of the methods described herein;

[0070] The memory is used to store the trained object detection model and detection results;

[0071] Image acquisition device, used to acquire monitoring images of the construction site;

[0072] An alarm device is used to issue an alarm signal when a safety violation is detected.

[0073] A computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described above.

[0074] The advantages and beneficial effects of this invention are as follows:

[0075] 1. This invention effectively improves the detection accuracy of small targets in construction site scenes by using SENetV2 multi-cardinality attention mechanism, Zoom_cat adaptive multi-scale fusion and ScalSeq3D sequence feature processing.

[0076] 2. The Zoom_cat module adopts a fusion of adaptive max pooling and average pooling, combined with nearest neighbor interpolation, which can better preserve the detailed and semantic information of multi-scale features. Compared with simple upsampling / downsampling, the detection accuracy is improved by about 0.84%.

[0077] 3. The ScalSeq module uses 3D convolution to process multi-scale feature sequences, which can effectively model the sequence relationship between scales and improve the feature representation capability. When used in combination with SENetV2, the detection accuracy is improved by about 1.3%.

[0078] 4. By utilizing the lightweight RepC3 module and the efficient RTDETRDecoder, detection accuracy is significantly improved while maintaining real-time inference speed, making it suitable for real-time applications such as construction site safety monitoring. Experimental results show that on an NVIDIA RTX 4090 GPU, the inference speed reaches 80 FPS, meeting the requirements of real-time video processing.

[0079] 5. By enhancing edge features and denoising training, the robustness of the model in complex construction site environments has been improved, enabling it to adapt to different lighting and weather conditions.

[0080] This invention achieves a technological breakthrough by constructing a multi-cardinality attention mechanism and a multi-scale feature fusion architecture. Its innovation is mainly reflected in the technical solutions corresponding to claims 4-6: First, the SENetV2 multi-cardinality attention mechanism uses four parallel fully connected layers to model channel dependencies, and its weight generation formula is as follows:

[0081] ;

[0082] This design breaks through the traditional single-path compression mode of SE modules; secondly, the Zoom_cat module achieves feature alignment through adaptive pooling and interpolation, and its fusion formula is as follows:

[0083] ;

[0084] The innovation of this technology lies in breaking through the conventional mindset of multi-scale fusion. Conventional methods such as FPN only use simple upsampling for feature alignment, leading to the loss of detailed information. This invention creatively combines adaptive max pooling with average pooling for downsampling, while simultaneously using nearest-neighbor interpolation for upsampling. This differentiated processing requires precise control over the degree of information preservation at different scales.

[0085] This addresses the issue of multi-scale feature information loss. Finally, the ScalSeq module innovatively transforms the scale dimension into the sequence dimension, and its 3D fusion formula is as follows:

[0086] ;

[0087] Conventional techniques only perform feature operations in the 2D spatial dimension, while this invention models the scale relationship as a 3D sequence problem and performs feature interaction in the scale dimension through a dedicated 3D convolution kernel with kernel_size=(1,1,1). This cross-dimensional feature recombination requires solving underlying problems such as tensor alignment and memory access optimization, and its technical implementation path is fundamentally different from conventional 2D convolution operations. Attached Figure Description

[0088] Figure 1This is a flowchart illustrating the overall process of the preferred embodiment of the method provided by the present invention. It mainly includes: site monitoring image acquisition and preprocessing; dataset creation and annotation; target detection network design and optimization; model training and validation; target recognition and safety monitoring;

[0089] Figure 2 This is a schematic diagram of the overall network architecture of the present invention, illustrating the complete process from input image to final detection result, including backbone network, neck network and detection head;

[0090] Figure 3 This is a diagram of the SENetV2 multi-cardinality attention mechanism, showing the structure of four parallel fully connected layers and the channel weight generation process;

[0091] Figure 4 This is a schematic diagram of Zoom_cat multi-scale feature fusion, showing the downsampling, upsampling and stitching process of three different scale features;

[0092] Figure 5 This is a flowchart of ScalSeq 3D sequence feature processing, which shows the complete process from 2D feature map to 3D tensor transformation, 3D convolution processing and output;

[0093] Figure 6 This is a schematic diagram of a depthwise separable convolution structure, illustrating the structure and convolution process of depthwise separable convolution. Detailed Implementation

[0094] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0095] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0096] A target detection method for construction site safety monitoring based on multi-scale feature fusion includes the following steps:

[0097] S1: Overall Technical Approach and Architecture Design. This invention uses a lightweight ResNet18 as the backbone network and enhances feature representation capabilities by embedding the SENetV2 multi-cardinality attention mechanism. In the feature fusion neck network, two complementary multi-scale feature fusion modules, Zoom_cat and ScalSeq, are designed to perform feature fusion from the spatial and sequence dimensions, respectively. The detection head adopts the RTDETRDecoder structure and achieves efficient target detection through a deformable Transformer decoder.

[0098] In this step, the overall architecture design philosophy is to ensure real-time performance through a lightweight backbone network, enhance feature representation capabilities through a multi-cardinality attention mechanism, and improve small object detection capabilities through a multi-layered feature fusion mechanism. Specifically, compared to the deeper ResNet50 / 101, ResNet18 reduces the number of parameters by approximately 70% and improves inference speed by approximately 2 times. At the same time, the SENetV2 attention mechanism compensates for the decrease in feature representation capabilities caused by the reduction in depth. The Zoom_cat and ScalSeq modules perform feature fusion from the spatial and sequence dimensions, respectively, which can fully utilize the complementarity of multi-scale information. The RTDETRDecoder adopts a deformable attention mechanism, which can better handle targets of different scales compared to traditional detection heads.

[0099] S2: Design Principles and Innovations of SENetV2's Multi-Cardimension Attention Mechanism. Traditional SE attention mechanisms use a single fully connected layer for channel compression, which can only learn one type of channel dependency, resulting in limited expressive power. In construction site safety monitoring scenarios, different channels may correspond to different semantic information (such as color, texture, shape, etc.), and a single fully connected layer is insufficient to fully model these complex inter-channel relationships.

[0100] In this step, the SENetV2 multi-cardinality attention mechanism proposed in this invention employs four parallel fully connected layers (cardinality=4). Each fully connected layer can learn channel dependencies in different dimensions, and through multi-cardinality fusion, it can capture richer inter-channel relationships. The innovation of this design lies in the fact that, through multiple parallel fully connected layers, it is possible to understand the inter-channel relationships from different perspectives, similar to the idea of ​​ensemble learning, where multiple weak learners combined can generate stronger expressive power.

[0101] The mechanism works as follows: First, spatial information is compressed into channel descriptors through global average pooling, preserving the global statistics of each channel. Then, four parallel fully connected layers learn channel dependencies in different dimensions, each analyzing the importance of a channel from a different perspective. Finally, the four outputs are concatenated and fused to generate more accurate channel weights. Compared to traditional SE mechanisms, the multi-cardinality design can understand the relationships between channels from multiple perspectives, improving feature representation capabilities.

[0102] The specific calculation formula is as follows:

[0103] ;

[0104] This formula compresses the spatial dimension through global average pooling to obtain global statistical information for each channel, which serves as the channel descriptor.

[0105] ;

[0106] Four parallel fully connected layers (k=1,2,3,4) respectively learn channel dependencies of different dimensions, and the ReLU activation function ensures the non-negativity of the output.

[0107] ;

[0108] ;

[0109] After concatenating the four outputs, pass them through a fully connected layer. Integrating multi-cardinality information, the Sigmoid activation function The output is mapped to the [0,1] interval and used as the channel weight.

[0110] ;

[0111] Finally, by multiplying element by element, channel weighting is applied to the original feature map to highlight important channels and suppress unimportant channels.

[0112] S3: The design principles and advantages of Zoom_cat adaptive multi-scale feature fusion. In construction site scenarios, targets of different scales require different levels of feature information: large targets (such as large equipment) require semantic information, while small targets (such as safety helmets and personnel) require detailed information. Traditional methods typically use simple upsampling or downsampling to unify the scale, which results in information loss: downsampling loses detailed information, and upsampling introduces artifacts.

[0113] In this step, the Zoom_cat module is designed to unify features from different scales to a medium scale through adaptive pooling fusion and nearest-neighbor interpolation, preserving both the semantic information of large-scale features and the detailed information of small-scale features. The advantages of this design are: adaptive pooling can dynamically adjust the pooling region according to the target size, better preserving important information compared to fixed-step pooling; and nearest-neighbor interpolation, compared to bilinear interpolation, does not introduce a smoothing effect, better preserving edge and detail information.

[0114] For large-scale features, an adaptive max-pooling and average-pooling fusion approach is used, which can simultaneously preserve both salient and average information of the features. Compared with single pooling, this fusion approach can retain richer information. For small-scale features, nearest-neighbor interpolation upsampling is used, which can avoid the smoothing effect of bilinear interpolation and better preserve detailed information. This design makes the fused features contain both rich semantic information and sufficient detailed information, making it particularly suitable for small object detection.

[0115] The specific calculation formula is as follows:

[0116] ;

[0117] Adaptive max pooling preserves salient features, while adaptive average pooling preserves average features. The combination of the two types of information is integrated.

[0118] ;

[0119] Nearest neighbor interpolation can avoid the smoothing effect of bilinear interpolation and better preserve edge and detail information.

[0120] ;

[0121] Features from three scales are spliced ​​along the channel dimension to form a fused multi-scale feature representation.

[0122] S4: Design Principles and Innovations of ScalSeq 3D Sequence Feature Processing. Traditional multi-scale feature fusion methods typically employ 2D convolutions, which cannot effectively model the sequence relationships between features at different scales. In deep learning, sequence relationships are usually modeled using RNNs, LSTMs, or Transformers, but these methods are computationally intensive and unsuitable for real-time applications.

[0123] In this step, the ScalSeq module proposed in this invention innovatively converts multi-scale features into 3D tensors and models the sequence relationships between scales through 3D convolution. The innovation of this design lies in treating the scale dimension as the sequence dimension and performing feature interaction in the scale dimension through 3D convolution, which enables the learning of dependencies between scales. At the same time, the computational cost of 3D convolution is smaller than that of RNN / LSTM, making it suitable for real-time applications.

[0124] The design philosophy of this module is to treat feature maps at three different scales (P3, P4, and P5) as a sequence. Through 3D convolution, feature interactions are performed along the scale dimension, enabling the learning of inter-scale dependencies. The third dimension of the 3D convolution corresponds to different scales, and convolution operations can fuse information from different scales to generate richer multi-scale feature representations. Specifically, a kernel_size of (1,1,1) indicates that convolution is not performed in the spatial dimension, but only in the scale dimension, thus specifically modeling the sequence relationships between scales.

[0125] The specific fusion formula is as follows:

[0126] ;

[0127] The formula states that: first, the feature maps at each scale are summed to obtain the aggregated features at that scale; then, the aggregated features from the three scales are fused using a Hadamard product, resulting in a feature map that incorporates multi-scale information. This design effectively models the sequence relationships between scales, improving the fusion effect of multi-scale features. Compared to simple addition or concatenation, the Hadamard product can better model the interaction relationships between scales.

[0128] Step 5: Design Principles and Selection Criteria for the Loss Function. This invention adopts the RTDETRDetectionLoss loss function, which includes three parts: classification loss, bounding box regression loss, and denoising training loss. The selection criteria for this design are as follows:

[0129] The classification loss employs Varifocal Loss, which, compared to traditional Focal Loss, handles class imbalance better. In construction site safety monitoring scenarios, the background area far exceeds the target area, resulting in severe class imbalance. Varifocal Loss, by dynamically adjusting the weights of easy and difficult samples, effectively addresses this imbalance.

[0130] The bounding box regression loss uses GIoU Loss. Compared to traditional IoU Loss, GIoU Loss considers the overlap area between the predicted and ground truth boxes, the distance between their centers, and the minimum bounding rectangle, providing better gradient signals and accelerating network convergence. Specifically, when the predicted and ground truth boxes do not overlap, the gradient of IoU Loss is 0, failing to provide an effective learning signal. GIoU Loss, by considering the minimum bounding rectangle, provides non-zero gradients, accelerating convergence.

[0131] Denoising training loss improves the robustness of the model by adding noise and performing denoising training during the training phase. Construction site environments are complex, with issues such as lighting changes, occlusion, and blurring; denoising training can enhance the model's robustness to these problems.

[0132] The specific calculation formula is as follows:

[0133] ;

[0134] Varifocal Loss can dynamically adjust the weights of easy and difficult samples, thus better handling the class imbalance problem.

[0135] ;

[0136] Compared to IoU Loss, GIoU Loss provides better gradient signals, accelerating network convergence.

[0137] ;

[0138] GIoU provides non-zero gradients even when the predicted bounding rectangle C does not overlap with the ground truth bounding rectangle, by considering the minimum bounding rectangle C.

[0139] ;

[0140] The total loss is the weighted sum of the three losses, and the importance of different losses can be balanced by adjusting the weights.

[0141] S6: Design Principles and Application Value of Edge Feature Enhancement. Addressing the problem of indistinct edge features for small targets in construction site scenarios, this invention proposes an edge feature enhancement method: fusing the edge-detected grayscale image with the original RGB image to form a four-channel input. The principle behind this design is that as the network depth increases, it primarily extracts semantic information from the image, gradually losing shallow edge information. Since small target detection mainly relies on edge information, explicitly providing edge information helps the network better recognize the contours of small targets.

[0142] The value of edge feature enhancement lies in the fact that targets such as safety helmets and people are mainly represented by edge contours in images. By explicitly providing edge information, the detection accuracy of these targets can be improved.

[0143] S7: Optimization Principles and Performance Analysis of Depthwise Separable Convolution. To improve inference speed while maintaining detection accuracy, this invention replaces part of the standard convolution with depthwise separable convolution. Depthwise separable convolution decomposes the standard convolution into two parts: channel-wise convolution and pointwise convolution, which can significantly reduce the amount of computation.

[0144] In this step, the optimization principle of depthwise separable convolution is as follows: standard convolution performs spatial convolution and channel fusion simultaneously, while depthwise separable convolution separates these two operations, performing channel-wise convolution first, followed by pointwise convolution, thus reducing computational cost. Specifically, in channel-wise convolution, each convolution kernel is responsible for only one channel, and the number of parameters is... Channel fusion is performed using pointwise convolution, with a parameter count of [number missing]. The total number of parameters is The standard convolution has 100,000 parameters. Therefore, the number of parameters in depthwise separable convolution is approximately that of standard convolution. times.

[0145] The specific calculation method is as follows:

[0146] ;

[0147] Compared to the computational cost of standard convolution The computational cost of depthwise separable convolution is approximately one-third that of standard convolution. This method significantly reduces computational load and increases inference speed while having only a slight impact on accuracy. It achieves a good balance between accuracy and speed.

[0148] This invention provides a target detection method for construction site safety monitoring based on multi-scale feature fusion, comprising the following steps:

[0149] S1, according to the flowchart Figure 1 This study expands upon existing publicly available construction site safety datasets. It also utilizes input layer edge feature enhancement. The original network input is a 3-channel RGB image. As the network depth increases, it primarily extracts semantic information. To improve the network's utilization of shallow edge features, the input layer was optimized. The core idea is to perform edge detection on the original RGB three-channel image, fuse the grayscale image obtained from the edge detection with the original RGB image's feature maps to form a four-channel image, and use this four-channel image as the data input.

[0150] The specific fusion process involves first dividing the RGB channels of the image into three single-channel images. Since a single-channel image is a two-dimensional matrix, a container can be used to sequentially store these three two-dimensional matrices. Since the grayscale image is also a single-channel image, the grayscale image containing edge features is then stored in the same container, forming a four-channel image. Edge detection uses the Canny edge detection algorithm, controlling the sensitivity of edge detection by setting low and high thresholds. The low threshold is set to 50, and the high threshold is set to 150, effectively extracting the edge contour information of targets such as safety helmets and personnel in construction site scenes.

[0151] S2 proposes optimizations and new model structures based on the existing network structure and its shortcomings. Figure 2 Compared to traditional object detection network structures, the improved network model optimizes the original deep network structure into a lightweight design, and reduces the backbone network from the original deep ResNet architecture to ResNet18. The feature fusion network adopts the lightweight RepC3 module and the efficient ScalSeq module, and the detection head adopts the RTDETRDecoder structure.

[0152] The reason for adopting a 16-layer RepC3 structure for the feature fusion network is that experiments have shown that 16 layers is the most concise number of network layers, and the 16-layer structure performs the best, ensuring both sufficient feature extraction and maintaining a relatively fast inference speed.

[0153] In current feature fusion networks, low-level features have high resolution and contain more location and detail information, while high-level features have stronger semantic information but low resolution and poor ability to perceive details. To improve the recognition ability of small targets, this invention employs a multi-scale feature fusion structure in the feature extraction network. The feature maps of three different scales (P3, P4, and P5) output from the backbone network are fused using the Zoom_cat and ScalSeq modules. The fused feature map is then input into the detection head for target detection.

[0154] The optimized model uses a combination of the Zoom_cat module and the ScalSeq module to perform multi-scale fusion of the P3, P4 and P5 scale feature maps output by the backbone network. Figure 4 The diagram shows the Zoom_cat module, which downsamples large-scale features through adaptive pooling and upsamples small-scale features through nearest neighbor interpolation, then stitches them together along the channel dimension after unifying them to a medium scale. Figure 5 This is a schematic diagram of the ScalSeq module, which converts feature maps of three different scales into 3D tensors and performs sequence feature modeling through 3D convolution, effectively improving the fusion effect of multi-scale features.

[0155] The improved model employs an AIFI module to perform internal scale feature interaction on the highest-level features, processes the feature sequence through a Transformer encoder, and preserves the multi-scale feature pyramid structure to detect objects at different scales. Its core idea is to fuse shallow and deep feature information to better utilize shallow features. The implementation process is as follows: To improve the utilization of detailed information from shallow networks, a feature layering method is used, with each layer predicting the detection result at its corresponding scale resolution. After lightweight optimization of the network, the multi-scale feature map output from the backbone network is fused with the feature map output from the feature fusion network, and this fusion is used as the input to the detection head.

[0156] It is the feature map output after ScalSeq feature fusion, and ⊙ represents the Hadamard product (element-wise multiplication). This refers to the i-th dimension feature map in the P3 scale feature map. This refers to the j-th dimension feature map in the P4 scale feature map. This refers to the k-th dimension feature map in the P5 scale feature map. Therefore, after ScalSeq feature fusion... It is a feature map that integrates multi-scale information.

[0157] To effectively improve the detection capability for small targets, the detection head adopts the RTDETRDecoder structure, and the number of queries is set to 300. Increasing the number of queries is used to improve the accuracy of small target detection. To further optimize the model, the loss function is changed from classification loss... Bounding box regression loss and denoising training loss It consists of three parts.

[0158] ;

[0159] Among them, the bounding box regression loss The core of the loss function is the bounding box regression loss, which adopts the GIoU loss function. The GIoU loss function takes into account three factors: the overlap area between the predicted box and the ground truth box, the distance between the center points, and the minimum bounding rectangle, which can improve the convergence speed and regression accuracy.

[0160] ;

[0161] When the predicted bounding box and the ground truth bounding box completely overlap, GIoU equals 1 and the loss is 0; when the predicted bounding box and the ground truth bounding box do not overlap, GIoU is less than IoU, which can provide better gradient signals and accelerate network convergence.

[0162] Figure 6 This diagram illustrates the optimization of depthwise separable convolution, primarily involving replacing some standard convolution operations with depthwise separable convolution. Depthwise separable convolution mainly consists of two parts: channel-wise convolution and pointwise convolution. Channel-wise convolution means each kernel is responsible for only one channel, mainly for filtering; pointwise convolution performs a weighted combination of the feature maps from the previous layer along the depth direction to generate new feature maps. Compared to standard convolution, depthwise separable convolution reduces the computational cost of convolution operations in the network model. The specific calculation method is as follows:

[0163] The computational cost of standard convolution for:

[0164] ;

[0165] Where W, H, and C are the length, width, and number of channels of the feature map. , It is the length and width of the convolution kernel. The number of convolution kernels.

[0166] With the same number of output feature maps, the computational cost of depthwise separable convolution is... for:

[0167] ;

[0168] Therefore, the computational cost of parameters for depthwise separable convolution is λ times that of standard convolution.

[0169] ;

[0170] When concatenation results are not concatenated, depthwise separable convolution has the same computational cost as standard convolution. This shows that as the number of convolution kernels increases, depthwise separable convolution can significantly reduce the computational cost of convolution operations.

[0171] Because the construction site safety monitoring dataset was not rich enough, the existing construction site dataset was expanded. The expanded dataset includes about 7,500 images containing construction site safety monitoring targets.

[0172] Optionally, to improve the small target detection capability, edge feature enhancement processing is performed on some images: the image is first processed using an image processing algorithm to obtain a grayscale image containing edge features, the image processing algorithm including grayscale conversion, binarization, erosion, dilation, denoising, edge detection or any combination thereof; then the channel image of the grayscale image containing edge features is fused with the original three-channel image to obtain a four-channel image containing edge features as input to the target detection model.

[0173] S3. Experiments were conducted using the improved network model on a construction site safety monitoring dataset. 7500 construction site safety monitoring images were randomly distributed, with 6000 images in the training set and 1500 images in the test set. During training, the number of iterations was set to 72 epochs, and the network model tended to stabilize when the maximum number of iterations was reached.

[0174] Table 1 (S4) shows the detection results compared to other network models; the method of this invention is an improved RT-DETR network model. To better evaluate these models, this invention selects commonly used image evaluation standards: mean accuracy (mAP) and the number of network model parameters. mAP is an important indicator of the quality of an object detection model; a higher mAP value indicates a higher average accuracy for object detection and better model performance. The number of network model parameters refers to the total number of parameters in the model; a smaller number of parameters results in faster network inference speed and is more suitable for real-time applications. Therefore, this invention comprehensively evaluates the model's performance from two aspects: accuracy and real-time performance.

[0175] S5, Some configurations for the embodiments. Embodiment 1: Network architecture configuration. This embodiment uses ResNet18 as the backbone network, with the following specific configuration: Backbone network configuration: Input layer: ConvNormLayer (3→32, kernel=3, stride=2); Initial convolutional layers: ConvNormLayer (32→32, kernel=3, stride=1) and ConvNormLayer (32→64, kernel=3, stride=1); Max pooling layer: MaxPool2d (kernel=3, stride=2, padding=1); Four residual stages: Stage 1: 2×BasicBlock_SENetV2 (64 channels), Stage 2: 2×BasicBlock_SENetV2 (128 channels), outputting P3 features, Stage 3: 2×BasicBlock_SENetV2 (256 channels), outputting P4 features, Stage 4: 2×BasicBlock_SENetV2 (512 channels), outputting P5 features. Each BasicBlock embeds an SENetV2 attention module, which employs a multi-cardinality (cardinality=4) channel attention mechanism with a compression ratio of reduction=16.

[0176] The neck network configuration includes: an AIFI module with 512 input channels, 1024 hidden dimensions, and 8 attention heads, which performs internal scale feature interaction on the highest-level features; a Zoom_cat module that receives P3, P4, and P5 scale features with 128, 256, and 512 input channels, respectively, and outputs 896 channels (128+256+512); a RepC3 module with 3 repetitions and a scaling factor of 0.5, employing a C3 bottleneck structure built using reparameterized convolutions; and a ScalSeq module with [128, 256, 512] input channels, which is unified to 256 channels through 1×1 convolutions, and outputs 256 channels.

[0177] Detection head configuration: RTDETRDecoder, 300 queries, 3 decoder layers, 8 attention heads, 256 hidden dimensions, generating target bounding boxes and class predictions through a deformable Transformer decoder.

[0178] Example 2: Training Parameter Settings. This example uses a construction site safety monitoring dataset for training. Specific parameters are as follows: Dataset: An expanded version of the publicly available construction site safety monitoring dataset, containing 7500 images covering different scenes and lighting conditions. Labeled categories include: safety helmets, people not wearing safety helmets, hazardous areas, safety equipment, and hazardous behaviors; Input size: 640×640 pixels; Batch size: 16; Number of training epochs: 72; Optimizer: AdamW, initial learning rate 0.0001, weight decay 0.0001; Learning rate scheduling: Cosine annealing strategy, minimum learning rate set to 0.01 times the initial learning rate; Loss function weights: λ1=2.0 (classification loss), λ2=5.0 (bounding box loss), λ3=1.0 (denoising loss).

[0179] Experimental results show that compared to YOLOv5, the present invention improves mAP by 4.1%, mAP by 4.6%, reduces the number of parameters by 10.9%, and increases inference speed by 37.0%; compared to YOLOv8, the present invention improves mAP by 1.6%, reduces the number of parameters by 4.7%, and increases inference speed by 1.4%; compared to Deformable DETR, mAP improves by 7.6%, reduces the number of parameters by 1.2%, and increases inference speed; compared to Conditional DETR, mAP improves by 7.2%, reduces the number of parameters by 6.8%, and increases inference speed. These data fully demonstrate the significant advantages of GF-RT-DETR in object detection tasks in terms of accuracy improvement and model efficiency optimization. Whether compared with the classic YOLO series models or with other advanced Transformer-based detection models, the present invention exhibits superior performance, proving its innovation and practicality in the field of object detection, and providing a more efficient and accurate object detection solution for practical application scenarios.

[0180] Example 3: Verification of the Feature Fusion Formula. This example verifies the effectiveness of the ScalSeq feature fusion formula proposed in claim 6. Experiments comparing different feature fusion methods: traditional concat fusion achieved an mAP of 70.8% and an mAP50 of 83.1%; simple add fusion achieved an mAP of 71.5% and an mAP50 of 84.3%; while the ScalSeq feature fusion formula of this invention achieved an mAP of 72.8% and an mAP50 of 85.2%. The experimental results show that using the ScalSeq feature fusion formula proposed in this invention significantly improves detection accuracy compared to traditional concat fusion, verifying the effectiveness of the formula.

[0181] Example 4: Validation of the SENetV2 Attention Mechanism. This example validates the effectiveness of the SENetV2 multi-cardinality attention mechanism. Experiments compare different attention mechanisms: no attention mechanism, mAP 70.1%, mAP50 83.5%; traditional SE attention mechanism, mAP 71.8%, mAP50 84.7%; the proposed SENetV2 multi-cardinality attention mechanism, mAP 72.7%, mAP50 85.4%. Experimental results show that the proposed SENetV2 multi-cardinality attention mechanism significantly improves detection accuracy compared to the no-attention mechanism, validating the effectiveness of the multi-cardinality attention mechanism.

[0182] This invention proposes a target detection method for construction site safety monitoring based on multi-scale feature fusion. By using SENetV2 multi-cardinality attention mechanism, Zoom_cat adaptive multi-scale fusion and ScalSeq 3D sequence feature processing, the accuracy of target detection in construction site safety monitoring is effectively improved, especially the detection accuracy of small targets, while maintaining real-time performance.

[0183] It should be noted that the user information (including but not limited to user device information, personal user information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the laws, regulations and standards of relevant countries and regions.

[0184] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0185] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A method for target detection in construction site safety monitoring based on multi-scale feature fusion, characterized in that, Includes the following steps: S1, preprocess the collected construction site monitoring images to generate training and test datasets; S2, Construct an object detection model, the input of which is the training data and the test data, and the output of which is the object recognition result; The training data is the data in the training dataset, and the test data is the data in the test dataset; The target detection model includes: a feature extraction backbone network, a feature fusion neck network, and a detection head network. The data output end of the feature extraction backbone network is connected to the data input end of the feature fusion neck network, and the data output end of the feature fusion neck network is connected to the data input end of the detection head network. The feature extraction backbone network adopts the ResNet18 architecture and embeds an SENetV2 multi-cardinality channel attention mechanism network attention module in each basic block. The SENetV2 attention module adopts a multi-cardinality channel attention mechanism to adaptively adjust the weights of the input features. The feature fusion neck network includes: an AIFI internal scale feature interaction module, used to perform internal scale feature interaction on the highest-level features output by the backbone network, flattening the feature map into a sequence and processing it through a Transformer encoder; a Zoom_cat adaptive multi-scale feature fusion module, used to receive feature maps of three different scales, downsample the largest-scale features using adaptive max pooling and average pooling, upsample the smallest-scale features using nearest-neighbor interpolation, unify them to a medium scale, and then concatenate them along the channel dimension; and a RepC3 lightweight network module based on reparameterized convolution. The block is used to construct a C3 bottleneck structure using reparameterized convolutions, and to perform lightweight processing on the fused features. The ScalSeq scale sequence feature processing module receives three feature maps P3, P4, and P5 at different scales. It unifies the number of channels through 1×1 convolutions, upsamples P4 and P5 to the P3 scale, converts the three feature maps into 3D tensors, processes them through 3D convolutions, batch normalization, and the LeakyReLU activation function, and finally outputs them through 3D max pooling and dimension compression. The output of the ScalSeq module is added element-wise to the P3 features processed by RepC3 through the Add addition module. The detection head network uses the RTDETRDecoder decoder to receive fused multi-scale features and generates target bounding boxes and class predictions through the deformable Transformer decoder. Training is performed using the RTDETRDetectionLoss loss function, which includes classification loss, bounding box regression loss, and denoising training loss. S3, Input the training data to train the target detection model; S4, Input the test data to test the target detection model.

2. The method for detecting construction site safety monitoring targets based on multi-scale feature fusion according to claim 1, characterized in that, The preprocessing of the acquired construction site monitoring images by S1 to generate training and testing datasets specifically includes: S1-1, collects construction site monitoring images, including safety helmet wearing detection, personnel behavior detection, dangerous area intrusion detection, and equipment status detection scenarios; S1-2, Use the annotation tool to annotate the target location information in each image to obtain the annotation file. The annotation categories include: safety helmet, person not wearing a safety helmet, dangerous area, safety equipment, and dangerous behavior; S1-3, Convert the format of the annotation file to obtain the dataset; S1-4 divides the dataset into a training dataset and a test dataset, with a ratio of 8:2 between the training dataset and the test dataset.

3. The method for detecting construction site safety monitoring targets based on multi-scale feature fusion according to claim 1, characterized in that, S1 further includes: The image is first processed using an image processing algorithm to obtain a grayscale image containing edge features. The image processing algorithm includes one or any combination of grayscale conversion, binarization, erosion, dilation, denoising, and edge detection. The grayscale image containing edge features is then fused with the original three-channel image to obtain a four-channel image containing edge features, which is used as input to the target detection model.

4. The method for detecting construction site safety monitoring targets based on multi-scale feature fusion according to claim 1, characterized in that, The specific implementation of the SENetV2 attention module includes: S2.1: Perform global average pooling on the input feature map to obtain channel descriptors; S2.2: Input the channel descriptors into four parallel fully connected layers respectively. Each fully connected layer compresses the number of channels to 1 / reduction of the original number of channels, where reduction=16; S2.3: The outputs of the four fully connected layers are concatenated along the channel dimension to obtain multi-cardinality fusion features; S2.4: The fused features are restored to the original number of channels through a fully connected layer, and channel weights are generated through the Sigmoid activation function; S2.5: Multiply the channel weights element-wise with the original feature map to obtain the weighted feature map.

5. The method for detecting construction site safety monitoring targets based on multi-scale feature fusion according to claim 1, characterized in that, The specific implementation of the Zoom_cat module includes: S3.1: Receive three input feature maps l, m, and s, representing large, medium, and small scales, respectively; S3.2: Downsample the large-scale feature l to a medium-scale m using adaptive max pooling and adaptive average pooling, and then add the two pooling results together; S3.3: Upsample the small-scale feature s to a medium-scale m using nearest neighbor interpolation; S3.4: Concatenate the processed l, m, and s feature maps along the channel dimension to output the fused feature map.

6. The method for detecting construction site safety monitoring targets based on multi-scale feature fusion according to claim 1, characterized in that, The specific implementation of the ScalSeq module includes: S4.1: Receive three feature maps P3, P4, and P5 of different scales, and unify the number of channels to a single channel by performing 1×1 convolution on each map. S4.2: Upsample P4 and P5 to the spatial dimensions of P3 using nearest neighbor interpolation; S4.3: Add a dimension to the third dimension of P3, P4_2, and P5_2 respectively, and convert them into 3D tensors; S4.4: Concatenate three 3D tensors along the third dimension to form a combined 3D tensor; S4.5: Perform 3D convolution (kernel_size=(1,1,1)), batch normalization, and LeakyReLU activation on the combined 3D tensor; S4.6: Process the data using 3D max pooling (kernel_size=(3,1,1)) and compress the third dimension to output the final feature map.

7. The method for detecting construction site safety monitoring targets based on multi-scale feature fusion according to claim 1, characterized in that, The RTDETRDetectionLoss loss function includes: S6.1: Classification Loss: The difference between the predicted class and the true class is calculated using the Varifoccal Loss function. The specific calculation formula is as follows: ； in To predict class probabilities, For real category labels, The number of samples; ; S6.2: Bounding Box Regression Loss: GIoU Loss is used to calculate the difference between the predicted and true bounding boxes. The specific calculation formula is as follows: ； in To predict the bounding box, For the true bounding box, The calculation formula is: ； in For intersection, union, and comparison, Let be the minimum bounding rectangle of the predicted bounding box and the ground truth bounding box. For the prediction box, For true bounding boxes; S6.3: Denoising Training Loss: During the training phase, the query with added noise is denoised and trained, and the loss between the denoised prediction and the true value is calculated. The total loss is: ； in This represents the training loss during denoising.

8. The method for detecting construction site safety monitoring targets based on multi-scale feature fusion according to claim 1, characterized in that, The target detection model also includes: Replacing some standard convolution operations with depthwise separable convolution, which includes channel-wise convolution and pointwise convolution; The term "channel-wise convolution" refers to the fact that each convolutional kernel is responsible for only one single channel, mainly for filtering. The pointwise convolution is a weighted combination of the feature maps of the previous layer in the depth direction to generate new feature maps; The specific calculation method for the depthwise separable convolution is as follows: ； in: This indicates the computational cost of depthwise separable convolution. Indicates the length of the feature map; Indicates the width of the feature map; Indicates the number of channels in the feature map; Indicates the length of the convolution kernel; Indicates the width of the convolution kernel; This indicates the number of convolution kernels.

9. A construction site safety monitoring target detection device based on multi-scale feature fusion, characterized in that, include: Processor, configured to perform the method according to any one of claims 1-8; The memory is used to store the trained object detection model and detection results; Image acquisition device, used to acquire monitoring images of the construction site; An alarm device is used to issue an alarm signal when a safety violation is detected.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

Images