Method for detecting unsafe behaviors of building workers based on Yolo

Abstract

The invention relates to the technical field of image processing, and discloses a construction worker unsafe behavior detection method based on Yolo, and the method comprises the steps: firstly obtaining an initial image of a construction site of a construction worker, carrying out the preprocessing of the initial image, and obtaining a to-be-detected image; performing feature extraction and unsafe behavior detection on the to-be-detected image through the target unsafe behavior detection model to obtain an unsafe behavior detection result of the building worker; according to the target unsafe behavior detection model, in a backbone network of a Yolo v11 model, a lightweight frequency dynamic convolution layer is used to replace a convolution layer in a second feature level and convolution layers in C3K2 modules in the second feature level and a third feature level, and a DSCA module is used to replace C3K2 modules in a fourth feature level and a fifth feature level. Training is carried out; according to the invention, small targets can be effectively detected under a complex background, the detection precision of unsafe behaviors of building workers is improved, and false detection and missing detection are avoided.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and relates to, but is not limited to, a method for detecting unsafe behaviors of construction workers based on YOLO. Background Technology

[0002] In recent years, serious accidents have been frequently caused by workers not wearing safety helmets or reflective vests at high-risk construction sites such as high-rise buildings, bridges, and tunnels. Currently, deep learning-based target detection technology can be used to monitor construction site safety. However, background noise from steel frames, cranes, machinery, and stacked materials at construction sites can easily interfere with the detection model's judgment, leading to false positives and false negatives. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method for detecting unsafe behaviors of construction workers based on YOLO, which can effectively detect small targets in complex backgrounds, improve the accuracy of unsafe behavior detection of construction workers, and avoid false detections and missed detections.

[0004] The specific technical solutions of this invention are as follows: This invention provides a method for detecting unsafe behaviors of construction workers based on YOLO. The method includes: Acquire initial images of the construction site where construction workers are working, and preprocess the initial images to obtain the images to be detected; The unsafe behavior detection model is used to extract features and detect unsafe behaviors in the image to be detected, thereby obtaining the unsafe behavior detection results of construction workers; wherein, the unsafe behavior detection results include at least: not wearing a safety helmet and not wearing a reflective vest; The target unsafe behavior detection model is obtained by replacing the convolutional layers in the second feature layer with lightweight frequency dynamic convolutional layers, and the convolutional layers in the C3K2 modules in the second and third feature layers with DSCA modules in the backbone network of the YOLO v11 model, and then training it. The lightweight frequency dynamic convolutional layer is used to generate dynamic convolutional kernels through frequency domain sparse weight learning and inverse Fourier transform to capture high-frequency texture information of small targets; the C3K2 module is used to enhance frequency domain features through residual branches and trunk branches; the DSCA module is used to extract and fuse features from the feature map output by the third feature layer by modeling spatial and channel dependencies.

[0005] In some embodiments, the lightweight frequency dynamic convolutional layer is specifically used to perform the following operations: Based on the input and output channels and kernel size of the lightweight frequency dynamic convolutional layer, the dimension of the theoretical convolutional kernel in the frequency domain is determined; all frequency coordinates are generated based on the input features through a two-dimensional discrete Fourier transform frequency network, and all frequency coordinates are sorted according to the Euclidean distance of each frequency coordinate to the origin. Attention weights are determined based on input features by an attention generator, and the attention weights are multiplied by the frequency domain parameters of the k preset base convolutional kernels in the lightweight frequency dynamic convolutional layer and summed along the kernel number dimension to obtain a frequency domain weight map. The real and imaginary parts of the frequency domain in the frequency domain weighted spectrum are filled into the corresponding frequency coordinates to obtain the discrete Fourier transform spectrum. The discrete Fourier transform spectrum is then restored to the spatial domain through inverse Fourier transform to obtain the dynamic convolution kernel. The input features are convolved using the dynamic convolution kernel to obtain a shallow enhanced feature map.

[0006] In some embodiments, the C3K2 module includes two lightweight frequency-dynamic convolutional layers, a split layer, at least two bottleneck units, and a splicing layer; wherein the at least two bottleneck units are arranged in series; the C3K2 module is specifically used to perform the following operations: The input features are subjected to lightweight frequency dynamic convolution through the first lightweight frequency dynamic convolution layer to obtain a frequency domain enhanced feature map; The frequency domain enhanced feature map is split into skip-preserving features and bottleneck input features through the splitting layer; The bottleneck output features are obtained by extracting features from the bottleneck input features using at least two bottleneck units. The bottleneck output feature and the skip retention feature are concatenated through the concatenation layer to obtain the concatenated feature; The spliced ​​features are subjected to lightweight frequency dynamic convolution through a second lightweight frequency dynamic convolution layer to obtain a mid-layer feature map.

[0007] In some embodiments, at least two of the bottleneck units include a first bottleneck unit and a second bottleneck unit; The step of extracting features from the bottleneck input features using at least two bottleneck units to obtain bottleneck output features includes: The bottleneck input feature is compressed and restored by the first bottleneck unit to obtain the first output feature, and the first output feature is residually connected with the bottleneck input feature to obtain the second output feature. The second bottleneck unit performs channel compression and channel recovery operations on the second output feature to obtain the third output feature, and then performs residual connection between the third output feature and the second output feature to obtain the bottleneck output feature.

[0008] In some embodiments, the bottleneck unit includes a first lightweight frequency dynamic convolutional layer and a second lightweight frequency dynamic convolutional layer; the first lightweight frequency dynamic convolutional layer is used to perform channel compression on the input features; the second lightweight frequency dynamic convolutional layer is used to perform channel restoration on the channel-compressed features.

[0009] In some embodiments, the DSCA module includes a batch normalization layer, a spatial modeling branch, a channel modeling branch, a gating fusion layer, and a feature fusion layer; The DSCA module is specifically used to perform the following operations: The input features are processed by a batch normalization layer to obtain a batch normalized feature map. The spatial modeling branch performs convolution in the height direction of the batch normalized feature map to obtain a vertical feature map, and performs convolution in the width direction of the batch normalized feature map to obtain a horizontal feature map. The vertical and horizontal feature maps are channel-compressed by the point-by-point convolutional layer in the channel modeling branch to obtain compressed vertical and compressed horizontal feature maps. The compressed vertical and compressed horizontal feature maps are then reshaped into vertical and horizontal attention feature maps with the same number of input feature channels. The vertical attention feature map and the compressed vertical feature map are dynamically weighted and fused through the gated fusion layer to obtain a vertical enhanced feature map; the horizontal attention feature map and the compressed horizontal feature map are dynamically weighted and fused to obtain a horizontal enhanced feature map. The feature fusion layer adds the vertically enhanced feature map and the horizontally enhanced feature map element by element to obtain a high-level feature map.

[0010] In some embodiments, the detection head of the target unsafe behavior detection model is a decoupled detection head; the decoupled detection head includes a classification branch and a bounding box regression branch; the classification branch is used to predict the category of small targets; The bounding box branch is used to determine the position and size of the small target and output the bounding box of the small target.

[0011] In some embodiments, the method further includes: A sample image set is obtained from a publicly available dataset of construction sites; wherein, the sample image set includes target categories and target bounding boxes; Based on the sample image set, the initial unsafe behavior detection model is iteratively trained to obtain the predicted category and predicted bounding box; The classification loss value between the target category and the predicted category is determined by the binary cross-entropy loss function, and the bounding box loss value between the target bounding box and the predicted bounding box is determined by the full intersection-over-union loss function, until the classification loss value and the bounding box loss value converge, thus obtaining the target unsafe behavior detection model.

[0012] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, an initial image of the construction site is first acquired, and the initial image is preprocessed to obtain the image to be detected. Then, a target unsafe behavior detection model is used to extract features and detect unsafe behaviors in the image to be detected, so as to obtain the unsafe behavior detection results of the construction workers. The unsafe behavior detection results include at least: not wearing a safety helmet and not wearing reflective clothing. The target unsafe behavior detection model is based on the YOLOv platform. In the backbone network of the v11 model, lightweight frequency dynamic convolutional layers replace the convolutional layers in the second feature layer, as well as the convolutional layers in the C3K2 modules in the second and third feature layers. DSCA modules replace the C3K2 modules in the fourth and fifth feature layers, and the model is trained accordingly. The lightweight frequency dynamic convolutional layers are used to generate dynamic convolutional kernels through frequency-domain sparse weight learning and inverse Fourier transform, capturing high-frequency texture information of small targets. The C3K2 modules are used for frequency-domain feature enhancement through residual branches and backbone branches. The DSCA module is used to extract and fuse features from the feature map output by the third feature layer by modeling spatial and channel dependencies. This enables effective detection of small targets in complex backgrounds, improving the accuracy of detecting unsafe behaviors of construction workers and avoiding false positives and false negatives. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 A flowchart illustrating the method for detecting unsafe behaviors of construction workers based on YOLO, as provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the target insecurity monitoring model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the C3K2LFD module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the Bottleneck LFD provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the DSCA module provided in an embodiment of the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0016] It should be noted that the terms "first, second, and third" used in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0017] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which these embodiments of the invention pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0018] Figure 1 A flowchart illustrating a method for detecting unsafe behaviors of construction workers based on Yolo is shown. This method can be executed via a control device, which may include at least one of a personal computer, laptop computer, smartphone, tablet computer, and portable wearable device; this embodiment is not limited to any particular type.

[0019] like Figure 1 As shown, the method for detecting unsafe behaviors of construction workers based on YOLO provided in this embodiment of the invention may include S101-S102.

[0020] S101. Obtain initial images of the construction site where construction workers are working, and preprocess the initial images to obtain the images to be detected.

[0021] In some embodiments, the control device acquires initial images of the construction site where construction workers are working, and preprocesses the initial images to obtain images to be detected.

[0022] For example, the control device can perform enhancement operations such as normalization, scaling, random flipping, and color jittering on the initial image to adapt it to the detection task and enhance the generalization of the model.

[0023] It should be noted that the embodiments of this application do not limit the preprocessing method of the initial image.

[0024] S102. The unsafe behavior detection model is used to extract features and detect unsafe behaviors in the image to be detected, and the results of the unsafe behavior detection of construction workers are obtained.

[0025] In some embodiments, the unsafe behavior detection results include at least: not wearing a helmet and not wearing a reflective vest.

[0026] For example, the control device uses a target unsafe behavior detection model to extract features and detect unsafe behaviors in the image to be detected, in order to determine whether the construction worker is not wearing a safety helmet or a reflective vest, thereby determining whether the construction worker is engaging in unsafe behavior and obtaining the unsafe behavior detection result of the construction worker.

[0027] Figure 2 A schematic diagram of the target unsafe behavior detection model is shown. In some embodiments, such as Figure 2 As shown, the target unsafe behavior detection model is obtained by replacing the convolutional layers in the second feature layer and the convolutional layers in the C3K2 modules of the second and third feature layers with lightweight frequency dynamic convolutional layers in the backbone network of the YOLO v11 model, and replacing the C3K2 modules in the fourth and fifth feature layers with dynamic spatial-channel attention (DSCA) modules. The lightweight frequency dynamic convolutional layers are used to generate dynamic convolutional kernels through frequency domain sparse weight learning and inverse Fourier transform, capturing high-frequency texture information of small targets. The C3K2 modules are used for frequency domain feature enhancement through residual branches and backbone branches. The DSCA module is used to extract and fuse features from the feature map output by the third feature layer by modeling spatial and channel dependencies.

[0028] In some embodiments, such as Figure 2As shown, the YOLO v11 model comprises a backbone network, a neck network, and a head network. The backbone network, the front-end of the YOLO v11 model, can include multiple feature layers. Each feature layer is used to extract features from the input image at different levels to capture semantic information from low to high levels. These feature layers can include a first feature layer, a second feature layer, a third feature layer, a fourth feature layer, and a fifth feature layer. The first feature layer performs preliminary feature extraction on the input image (i.e., the image to be detected), obtaining a shallow feature map. The second and third feature layers are used for further feature extraction to obtain a mid-level feature map. The fourth and fifth feature layers are used to extract high-level semantic information, obtaining a high-level feature map. In other words, the backbone network is mainly used to extract basic features from the input image (equivalent to the image to be detected), transforming the original image into feature maps of different scales through operations such as convolution and pooling, providing basic features for subsequent feature fusion and recognition.

[0029] It should be noted that in the traditional C3K2 module, C3 represents the convolutional block with residual connections, and K2 indicates that a 2×2 convolutional kernel is used within the module. In this embodiment, lightweight frequency-dynamic convolutional layers replace the convolutional layers in the C3K2 module in the second and third feature levels. This is to distinguish it from the traditional C3K2 module. Figure 2 The C3K2 module that was replaced by the C3K2 module is called the C3K2LFD module.

[0030] In some embodiments, such as Figure 2 As shown, the backbone network of the target insecurity detection model includes a Convolutional Block with Shortcut (CBS) module, a Lightweight-Frequency-Dynamic Convolution (LFDConv) layer, a C3K2LFD module, a CBS module, a C3K2LFD module, a CBS module, a DSCA module, a CBS module, a DSCA module, a Spatial Pyramid Pooling Fast (SPPF) module, and a Cross-Channel Partial Spatial Attention (C2PSA) module.

[0031] For example, such as Figure 2As shown, the control device inputs the image to be detected (RGB, 640×640) into the target insecurity detection model. The first CBS module performs a downsampling convolution with a stride of 2 on the image to be detected and uses a shortcut mechanism to retain some original features to obtain a shallow feature map with preliminary low-level texture and edge information. The shallow feature map is then input into a lightweight frequency dynamic convolution layer.

[0032] In some embodiments, the lightweight frequency dynamic convolutional layer specifically performs the following operations: determining the dimension of the theoretical convolutional kernel in the frequency domain based on the input and output channels and kernel size of the lightweight frequency dynamic convolutional layer; generating all frequency coordinates based on the input features using a two-dimensional discrete Fourier transform frequency network (2DFFT frequency grid), and sorting all frequency coordinates according to their Euclidean distance from the origin; determining attention weights based on the input features using an attention generator, and multiplying the attention weights by the frequency domain parameters of the k preset base convolutional kernels in the lightweight frequency dynamic convolutional layer and summing them along the kernel number dimension to obtain a frequency domain weight map; filling the real and imaginary parts of the frequency domain in the frequency domain weight map into the corresponding frequency coordinates to obtain a discrete Fourier transform spectrum, and restoring the discrete Fourier transform spectrum to the spatial domain using an inverse Fourier transform to obtain a dynamic convolutional kernel; and convolving the input features using the dynamic convolutional kernel to obtain a shallow enhanced feature map.

[0033] Here, the input features of the lightweight frequency dynamic convolutional layer are the shallow feature map output by the first CBS module. After receiving the shallow feature map, the lightweight frequency dynamic convolutional layer first calculates the dimension of the theoretical convolution kernel in the frequency domain based on the input and output channels and the kernel size. Then, it generates all frequency coordinates through a two-dimensional discrete Fourier transform frequency network (i.e., torch.fft.fftfreq and rfftfreq), and sorts all frequency coordinates according to their Euclidean distance from the origin. This ensures that low-frequency information is preserved first, while providing an index for subsequent frequency domain weight filling. Next, a lightweight attention generator (e.g., k_att) performs global average pooling on the shallow feature map to extract channel-level global descriptions. Then, a multilayer perceptron (MLP) or a 1×1 convolutional layer (Conv) performs a non-linear transformation on the pooled shallow feature map to generate a weight vector (i.e., attention weights) corresponding to each of the K preset base convolutional kernels in the lightweight frequency dynamic convolutional layer. Normalized exponential functions (softmax functions) or temperature-scaled normalization can be used to ensure that the sum of the attention weights of each base convolutional kernel is controlled. Each sample (small target) in the shallow feature map corresponds to a set of convolutional kernel weights. After obtaining the attention weights of each base convolutional kernel, these attention weights are multiplied by the frequency domain parameters of the k preset base convolutional kernels in the lightweight frequency dynamic convolutional layer and summed along the kernel number dimension to obtain the frequency domain weight map for each sample. Then, the real and imaginary parts of the frequency domain weight map are filled into the corresponding frequency coordinates to obtain the Discrete Fourier Transform (DFT) spectrum. Next, the DFT spectrum is restored to the spatial domain using a two-dimensional inverse real fast Fourier transform (IRFT2, equivalent to the inverse Fourier transform above), yielding the dynamic convolutional kernel weights that can actually be used for convolution, thus obtaining the dynamic convolutional kernel. This makes the convolutional kernel no longer statically fixed but adaptively generated based on the input image features. Finally, the obtained dynamic convolutional kernel is used to perform a standard 2D convolution on the shallow feature map to obtain the shallow enhanced feature map. The convolution parameters (such as stride and padding) can be kept consistent with the original YOLO v11 model configuration. These steps complete the mapping from input features to output features, while also incorporating frequency domain augmentation and sample adaptation mechanisms.

[0034] Understandably, replacing the convolutional layers in the second feature layer with lightweight, frequency-dynamic convolutional layers allows the target insecurity detection model to incorporate frequency domain modeling as early as possible during feature extraction. Dynamic convolutional kernels are generated through frequency-domain sparse weight learning and inverse FFT. This enables the target insecurity detection model to better capture high-frequency texture information of small targets against complex backgrounds. Thus, the target insecurity detection model no longer relies solely on spatial convolution stacking but achieves coordinated feature enhancement across the frequency and spatial domains.

[0035] Figure 3 A schematic diagram of the C3K2 module is shown. In some embodiments, such as... Figure 3 As shown, the C3K2 module includes two lightweight frequency dynamic convolutional layers (LFDConv), a split layer, at least two bottleneck units, and a concatenation layer. The at least two bottleneck units are connected in series. The number of bottleneck units can be two or more; however, this embodiment does not limit the number of bottleneck units. Figure 3 The example is illustrated by using two bottleneck units.

[0036] It should be noted that, since the embodiments of this application use lightweight frequency dynamic convolutional layers to replace the convolutional layers in the bottleneck unit, in order to distinguish it from the traditional bottleneck unit, Figure 3 The bottleneck unit that was replaced by the Chinese general is called BottleneckLFD.

[0037] In some embodiments, the C3K2 module is specifically used to perform the following operations: performing lightweight frequency dynamic convolution on the input features through a first lightweight frequency dynamic convolution layer to obtain a frequency domain enhanced feature map; splitting the frequency domain enhanced feature map into skip-preserving features and bottleneck input features through a splitting layer; extracting features from the bottleneck input features through at least two bottleneck units to obtain bottleneck output features; concatenating the bottleneck output features and skip-preserving features through a concatenation layer to obtain concatenated features; and performing lightweight frequency dynamic convolution on the concatenated features through a second lightweight frequency dynamic convolution layer to obtain a mid-layer feature map.

[0038] Here, the input features of the first lightweight frequency-dynamic convolutional layer are the shallow enhanced feature maps output by the lightweight frequency-dynamic convolutional layer. For example, after passing through the first lightweight frequency-dynamic convolutional layer, the shallow enhanced features are split in two by a splitting layer, forming two parallel branches. One branch serves as the preserved path for skip connections, while the other branch enters the bottleneck unit, i.e., the bottleneck input features are input into the bottleneck unit. The bottleneck unit consists of two lightweight frequency-dynamic convolutional layers. The first lightweight frequency-dynamic convolutional layer compresses the number of channels to one-quarter of the original number, and the second lightweight frequency-dynamic convolutional layer restores the number of channels to half of the original number, followed by residual connections. The C3K2 module can have multiple bottleneck units. Features are output after passing through each bottleneck unit. After passing through all the concatenated bottleneck units, the output features as preserved paths (i.e., skip-preserved features) are concatenated with the output features of the bottleneck units (i.e., bottleneck output features). The number of channels in the concatenated multi-path features increases, so they are integrated using a 1×1 convolution (i.e., a lightweight frequency-dynamic convolutional layer).

[0039] In some embodiments, such as Figure 3 As shown, at least two bottleneck units include a first bottleneck unit ( Figure 3 The first Bottleneck LFD from top to bottom) and the second bottleneck unit ( Figure 3 The second Bottleneck LFD (from top to bottom) extracts bottleneck output features by using at least two bottleneck units to process the bottleneck input features. This process includes: performing channel compression and channel restoration operations on the bottleneck input features using the first bottleneck unit to obtain the first output feature, and then performing a residual concatenation between the first output feature and the bottleneck input features to obtain the second output feature; performing channel compression and channel restoration operations on the second output feature using the second bottleneck unit to obtain the third output feature, and then performing a residual concatenation between the third output feature and the second output feature to obtain the bottleneck output feature.

[0040] Figure 4 A schematic diagram of the bottleneck unit is shown. In some embodiments, such as... Figure 4 As shown, the bottleneck unit includes a first lightweight frequency dynamic convolutional layer ( Figure 4 The first LFDConv from top to bottom) and the second lightweight frequency dynamic convolutional layer ( Figure 4 The second LFDConv (from top to bottom) is used to perform channel compression on the input features; the first lightweight frequency dynamic convolutional layer is used to perform channel restoration on the channel-compressed features.

[0041] Understandably, replacing all convolutional layers in the C3K2 module with lightweight frequency-dynamic convolutional layers allows the C3K2 module to retain the original residual and bottleneck design of the traditional C3K2 module while enhancing its frequency domain representation capabilities. Compared to the traditional C3K2 module, it can more effectively capture small target features and texture details in complex backgrounds, improving the detection accuracy of unsafe behaviors.

[0042] Figure 5 A schematic diagram of the DSCA module is shown. In some embodiments, such as... Figure 5 As shown, the DSCA module includes a batch normalization layer (BatchNorm), a spatial modeling branch, a channel modeling branch, a gated fusion layer, and a feature fusion layer.

[0043] For example, the spatial modeling branch includes a horizontal convolutional branch and a vertical convolutional branch, both of which can be deep convolutional layers (DWConv). The channel modeling branch includes a pointwise convolutional layer (PWConv) and a multi-head dynamic normalization layer; the feature fusion layer can include a cross-attention 3D object detection with dynamic fusion (DyFusion) layer and a pointwise convolutional layer.

[0044] In some embodiments, the DSCA module is specifically used to perform the following operations: batch normalize the input features through a batch normalization layer to obtain a batch normalized feature map; and perform spatial modeling branching on the height direction of the batch normalized feature map (…). Figure 5 (Illustrated by H) Convolution is performed to obtain a vertical feature map, and then the width direction of the batch normalized feature map ( Figure 5 (Illustrated by W) Convolution is performed to obtain a horizontal feature map; the vertical and horizontal feature maps are compressed through pointwise convolutional layers in the channel modeling branch to obtain compressed vertical and compressed horizontal feature maps, which are then reshaped into vertical and horizontal attention feature maps with the same number of input feature channels; the vertical attention feature map and compressed vertical feature map are dynamically weighted and fused through a gated fusion layer to obtain a vertical enhanced feature map; the horizontal attention feature map and compressed horizontal feature map are dynamically weighted and fused to obtain a horizontal enhanced feature map; the vertical enhanced feature map and horizontal enhanced feature map are element-wise added through a feature fusion layer to obtain a high-level feature map.

[0045] Here, the mid-level feature map output by the C3K2 module is passed through the downsampled convolution of the third CBS module, and the input features of the batch normalization layer are the features output by the third CBS module. In practical applications, let the input features be X∈R. B×C×H×W Where X is the input feature map, R indicates that all elements of X are real numbers, B is the batch size, C is the number of channels, and H and W are the height and width of the feature map. After the input features enter the DSCA module, they first pass through a batch normalization layer for edge transformation processing to obtain the standardized features X. norm =BN(X). Then, the spatial modeling branch applies the standardized features X... norm Spatial feature extraction is performed by convolution in the height direction (1×1 kernel) to extract feature X. norm The texture and structural features in the vertical direction are used to obtain a vertical feature map V; simultaneously, convolution (1×1 kernel) is performed in the width direction to capture features X. norm The horizontal edge and contour information is used to obtain the horizontal feature map H. After spatial feature extraction, the channel modeling branch uses a point-wise convolutional layer (PWConv) to compress the number of channels in both the vertical and horizontal feature maps to an intermediate dimension, resulting in the compressed vertical feature map V. l And compress the lateral feature map H1; then, compress the longitudinal feature map V. l The compressed horizontal feature map H1 is normalized using a multi-head grouping method, which means that the compressed vertical feature map V... l The compressed lateral feature map H1 is divided into a predetermined number of subspaces to obtain new feature maps. A softmax function is applied to the features in each subspace to obtain the weight distribution of the features in each subspace. The features in each subspace are then weighted and summed according to the weight distribution to obtain a weighted feature map. Next, the weighted feature maps of each subspace are concatenated to restore the original feature map with the same number of channels as the input feature map, resulting in a vertical attention feature map and a lateral attention feature map. For the features of each directional branch (vertical and lateral attention feature maps), the original feature maps (compressed vertical and compressed lateral feature maps) are fused with the attention enhancement features (vertical and lateral attention feature maps) through a learnable gating parameter λ to obtain a vertical enhancement feature map and a lateral enhancement feature map; that is, through Y=λX dn +(1−λ)X in The original feature map is fused with attention-enhanced features. Here, Y represents the output feature, and X represents the input feature. dn For attention-enhancing features, X inThe original feature map is λ ∈ [0,1], initialized to 0.5. During training, λ can be adaptively updated. The closer λ is to 1, the more the model relies on attention to enhance features; the closer λ is to 0, the more the model retains the original feature map. Finally, the vertically enhanced feature map and the horizontally enhanced feature map are added element-wise to obtain the high-level feature map.

[0046] It is understandable that the DSCA module replaces the C3K2 module in the YOLO v11 model. Since the DSCA module can model both spatial and channel dependencies simultaneously, it uses vertical and horizontal separable convolutions combined with a multi-head attention mechanism to dynamically allocate spatial / channel weights. Therefore, compared to the traditional YOLO v11 model, the target insecurity detection model can dynamically model the interaction between spatial and channel dimensions, thereby achieving target region enhancement and background noise suppression.

[0047] In some embodiments, the neck network is the middle section of the YOLO v11 model, mainly used for feature fusion. The neck network receives feature maps of different scales output by the backbone network and achieves multi-scale feature fusion through operations such as upsampling, downsampling, and feature concatenation. In this embodiment, the target insecurity detection model maintains the Feature Pyramid Network (FPN) + Path Aggregation Network (PAN) structure of the YOLO v11 model unchanged, forming a joint representation of high, medium, and low-level features. Since the neck network of the YOLO v11 model is prior art, it will not be described in detail here.

[0048] Understandably, during the fusion process, the enhanced high-frequency texture features of LFDConv and the semantic features weighted by DSCA are fused at multiple scales, which can ensure that targets at different resolutions in the feature map after the neck network fusion can be accurately perceived.

[0049] In some embodiments, the head network is the latter part of the YOLO v11 model. The head network includes a detection head, which is mainly used for detection. After receiving the feature map fused by the neck network, the head network can predict the category and location information of the target in the image through convolution operations and output the detection results.

[0050] In some embodiments, the detection head of the target unsafe behavior detection model is a decoupled detection head; the decoupled detection head includes a classification branch and a bounding box regression branch.

[0051] The classification branch is used to predict the category of the small target, and the bounding box branch is used to determine the location and size of the small target and output the bounding box of the small target.

[0052] Understandably, the classification branch relies on highly semantic features weighted by DSCA, which helps to accurately identify small object categories in complex backgrounds. The regression branch relies on frequency domain enhanced features to improve the localization accuracy and robustness of the bounding box.

[0053] The YOLO-based method for detecting unsafe behaviors of construction workers provided in this application further includes: obtaining a sample image set from a publicly available dataset of construction sites; wherein the sample image set includes target categories and target bounding boxes; iteratively training an initial unsafe behavior detection model based on the sample image set to obtain predicted categories and predicted bounding boxes; determining the classification loss value between the target category and the predicted category through the binary cross-entropy loss function (BCE Loss), and determining the bounding box loss value between the target bounding box and the predicted bounding box through the complete intersection over union loss function (CIOU Loss), until the classification loss value and the bounding box loss value converge to obtain the target unsafe behavior detection model.

[0054] In some embodiments, the publicly available construction site dataset SODA (Site Object Detection Dataset) can be used as the training dataset. This training dataset includes images of 15 target categories, such as workers, materials, and machines, collected from different sites, weather conditions, and angles. Images containing people, safety helmets, and reflective vests are then selected from the training dataset as a sample image set. This sample image set is then divided into a training set, a test set, and a validation set in a 7:2:1 ratio. Finally, the initial unsafe behavior detection model is trained using the training set, its performance is tested using the test set, and its performance is validated and optimized using the validation set. After validation and optimization, the target unsafe behavior detection model is obtained.

[0055] Understandably, using the binary cross-entropy loss function in the classification branch can address the class imbalance problem for small objects, while using the full intersection-union loss function in the bounding box regression branch can improve bounding box accuracy. Regularization constraints are added to the frequency domain weights of LFDConv to avoid numerical instability in the early stages of training. The AdamW optimizer and Cosine learning rate scheduling are employed to improve training stability and convergence speed.

[0056] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the invention, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the invention. The sequence numbers of the above-described embodiments of the invention are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0057] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0058] In the several embodiments provided by this invention, it should be understood that the disclosed methods can be implemented in other ways. The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0059] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A Yolo-based construction worker unsafe behavior detection method, characterized in that, The method comprises: obtaining an initial image of a construction site of a construction worker, preprocessing the initial image to obtain a to-be-detected image; performing feature extraction and unsafe behavior detection on the to-be-detected image through a target unsafe behavior detection model to obtain an unsafe behavior detection result of the construction worker; wherein the unsafe behavior detection result at least includes: not wearing a safety helmet and not wearing a reflective vest; The target unsafe behavior detection model is obtained by replacing the convolution layer in the second feature level, the convolution layer in the C3K2 module in the second feature level and the third feature level, the C3K2 module in the fourth feature level and the fifth feature level with a light frequency dynamic convolution layer in the backbone network of the Yolo v11 model, and replacing the C3K2 module in the fourth feature level and the fifth feature level with a DSCA module, and training; The light frequency dynamic convolution layer is used to generate a dynamic convolution kernel through frequency domain sparse weight learning and inverse Fourier transform to capture small target high frequency texture information; the C3K2 module is used to perform frequency domain feature enhancement through a residual branch and a main branch; and the DSCA module is used to model space dependence and channel dependence to perform feature extraction and fusion on the feature map output by the third feature level.

2. The method of claim 1, wherein, The light frequency dynamic convolution layer is specifically used to perform the following operations: According to the input and output channels and the convolution kernel size of the light frequency dynamic convolution layer, the dimension of the theoretical convolution kernel in the frequency domain is determined; all frequency coordinates are generated based on the input features through a two-dimensional discrete Fourier transform frequency network, and all frequency coordinates are sorted according to the Euclidean distance of each frequency coordinate to the origin; An attention generator determines attention weights based on the input features, multiplies the attention weights with the frequency domain parameters of the preset k base convolution kernels in the light frequency dynamic convolution layer, and sums along the kernel number dimension to obtain a frequency domain weight atlas; The real part and the imaginary part of the frequency domain in the frequency domain weight atlas are filled into the corresponding frequency coordinates to obtain a discrete Fourier transform spectrum, and the discrete Fourier transform spectrum is restored to the spatial domain through inverse Fourier transform to obtain a dynamic convolution kernel; The input features are convolved through the dynamic convolution kernel to obtain a shallow layer enhanced feature map.

3. The method of claim 2, wherein, The C3K2 module comprises two light frequency dynamic convolution layers, a split layer, at least two bottleneck units and a splicing layer; wherein the at least two bottleneck units are connected in series; and the C3K2 module is specifically used to perform the following operations: The first light frequency dynamic convolution layer is used to perform light frequency dynamic convolution on the input features to obtain a frequency domain enhanced feature map; The split layer is used to split the frequency domain enhanced feature map into a jump reserved feature and a bottleneck input feature; The at least two bottleneck units are used to extract features from the bottleneck input feature to obtain a bottleneck output feature; The splicing layer is used to splice the bottleneck output feature and the jump reserved feature to obtain a spliced feature; The second light frequency dynamic convolution layer is used to perform light frequency dynamic convolution on the spliced feature to obtain a middle layer feature map.

4. The method of claim 3, wherein, The at least two bottleneck units comprise a first bottleneck unit and a second bottleneck unit; The feature extraction on the bottle neck input feature through the at least two bottle neck units obtains a bottle neck output feature, and the feature extraction includes: The first bottle neck unit is configured to perform channel compression and channel recovery operations on the bottle neck input feature to obtain a first output feature, and perform residual connection between the first output feature and the bottle neck input feature to obtain a second output feature. The second bottle neck unit is configured to perform channel compression and channel recovery operations on the second output feature to obtain a third output feature, and perform residual connection between the third output feature and the second output feature to obtain the bottle neck output feature.

5. The method according to claim 3 or 4, characterized in that, The bottle neck unit includes a first lightweight frequency dynamic convolution layer and a second lightweight frequency dynamic convolution layer; the first lightweight frequency dynamic convolution layer is configured to perform a channel compression operation on an input feature; and the second lightweight frequency dynamic convolution layer is configured to perform a channel recovery operation on the feature after the channel compression.

6. The method of claim 1, wherein, The DSCA module includes a batch normalization layer, a spatial modeling branch, a channel modeling branch, a gate fusion layer, and a feature fusion layer. The DSCA module is specifically configured to perform the following operations: The batch normalization layer is configured to perform batch normalization processing on an input feature to obtain a batch normalization feature map. The spatial modeling branch is configured to perform convolution on the batch normalization feature map in a height direction to obtain a vertical feature map, and perform convolution on the batch normalization feature map in a width direction to obtain a horizontal feature map. The point-wise convolution layer in the channel modeling branch is configured to perform channel compression on the vertical feature map and the horizontal feature map to obtain a compressed longitudinal feature map and a compressed transverse feature map, and reshape the compressed longitudinal feature map and the compressed transverse feature map into a longitudinal attention feature map and a transverse attention feature map with the same number of channels as the input feature. The gate fusion layer is configured to perform dynamic weighted fusion on the longitudinal attention feature map and the compressed longitudinal feature map respectively to obtain a longitudinal enhanced feature map, and perform dynamic weighted fusion on the transverse attention feature map and the compressed transverse feature map to obtain a transverse enhanced feature map. The feature fusion layer is configured to perform element-wise addition on the longitudinal enhanced feature map and the transverse enhanced feature map to obtain a high-level feature map.

7. The method of claim 1, wherein, The detection head of the target unsafe behavior detection model is a decoupled detection head; the decoupled detection head includes a classification branch and a bounding box regression branch; the classification branch is configured to predict the class of a small target; The bounding box branch is configured to determine the position and size of the small target, and output the bounding box of the small target.

8. The method of claim 1, wherein, The method further includes: Obtaining a sample image set from a construction site public data set; wherein the sample image set includes a target class and a target bounding box; Iteratively training an initial unsafe behavior detection model based on the sample image set to obtain a predicted class and a predicted bounding box; Determining a classification loss value between the target class and the predicted class through a binary cross-entropy loss function, and determining a bounding box loss value between the target bounding box and the predicted bounding box through a complete intersection over union loss function, until the classification loss value and the bounding box loss value converge, to obtain the target unsafe behavior detection model.

Images