Multi-target wearing identification method based on PHSM-YOLO

Abstract

The invention discloses a PHSM-YOLO-based multi-target wearing identification method. The PHSM-YOLO-based multi-target wearing identification method comprises the following steps: constructing a PHSM-YOLO target detection model used for identifying whether a constructor correctly wears a safety appliance; collecting a target image containing constructors in the construction environment; and inputting the target image into a PHSM-YOLO target detection model, and outputting a wearing identification result of the safety wearing tool of each construction worker in the target image by the PHSM-YOLO target detection model. According to the multi-target wearing identification method, the wearing of the safety wearing tool can be identified on the target image containing the construction personnel through the constructed PHSM-YOLO target detection model, so that the personal safety of the construction personnel is guaranteed.

Description

Technical Field

[0001] This invention relates to a multi-target wearable recognition method, and more particularly to a multi-target wearable recognition method based on PHSM-YOLO. Background Technology

[0002] With the rapid development of artificial intelligence, the concepts of smart cities and safe China are deeply integrated with AI, bringing widespread attention to the safety of construction workers. However, severe occlusion, dense targets, and background interference all seriously affect the effectiveness of existing target detection methods. Therefore, researching multi-target detection methods in complex environments and improving the detection accuracy of safety equipment worn by construction workers is of great significance for ensuring construction safety and the personal safety of workers.

[0003] As deep learning continues to improve its performance in handling complex tasks, it has also provided a simple and efficient method for detecting safety equipment. Deep learning-based object detection algorithms, such as two-stage algorithms like R-CNN and Faster R-CNN, and one-stage algorithms like YOLO and SSD, demonstrate unique advantages over traditional machine learning algorithms. They overcome some limitations of traditional algorithms, do not have overly strict requirements on the amount of training data, and exhibit superior feature extraction capabilities. Convolutional neural networks can autonomously learn and extract higher-level, more expressive features, enhancing processing efficiency and significantly improving the speed of information transmission, thereby improving the accuracy of real-time detection. The former method has the advantage of high accuracy, enabling precise location and classification of safety equipment, and high robustness to complex backgrounds. However, because it requires two-step detection, the detection speed is relatively slow and cannot well meet the needs of real-time detection. The latter detection method's most obvious advantage is its speed and ease of model implementation, allowing it to perform excellently in safety equipment detection tasks. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-target wearable recognition method based on PHSM-YOLO, which can solve the problem of detecting and recognizing the wear of safety equipment on multiple targets in complex construction environments, thereby ensuring the personal safety of construction workers.

[0005] Technical solution: The multi-target wearable recognition method based on PHSM-YOLO of the present invention is characterized by the following steps:

[0006] Step 1: Construct a PHSM-YOLO target detection model to identify whether construction workers are wearing safety equipment correctly;

[0007] Step 2: Acquire target images containing construction workers in the construction environment;

[0008] Step 3: Input the target image into the PHSM-YOLO target detection model, and the PHSM-YOLO target detection model will output the recognition results of the safety equipment worn by each construction worker in the target image.

[0009] Furthermore, in step 1, the constructed PHSM-YOLO target detection model includes a backbone network, a neck network, and a detection head;

[0010] The backbone network comprises, in sequence, a first Conv module, a second Conv module, a first C3k2_PPA module, a first HWD module, a second C3k2_PPA module, a second HWD module, a third C3k2_PPA module, a third HWD module, a fourth C3k2_PPA module, an SPPF module, a C2PSA module, and an MPCA module; the input terminal of the first Conv module serves as the input terminal of the backbone network, used to input the target image;

[0011] The neck network includes a first Concat module, a second Concat module, a third Concat module, a fourth Concat module, a first C3K2 module, a second C3K2 module, a third C3K2 module, a fourth C3K2 module, a fourth HWD module, a fifth HWD module, a first Upsample module, and a second Upsample module;

[0012] The output data of the second C3k2_PPA module is also input into the first Concat module. The first Concat module, the first C3K2 module, the fourth HWD module, the second Concat module, the second C3K2 module, the fifth HWD module, the third Concat module, and the third C3K2 module are connected in series.

[0013] The output data of the third C3k2_PPA module is also input into the fourth Concat module. The output data of the fourth Concat module is input into the fourth C3K2 module. The output data of the fourth C3K2 module is input into the first Upsample module and the second Concat module respectively. The output data of the first Upsample module is input into the first Concat module.

[0014] The output data of the MPCA module is input into the second Upsample module and the third Concat module respectively, and the output data of the second Upsample module is input into the fourth Concat module.

[0015] The detection head includes a first SEAMHead module, a second SEAMHead module, and a third SEAMHead module. The output data of the first C3K2 module is also input into the first SEAMHead module, the output data of the second C3K2 module is also input into the second SEAMHead module, and the output data of the third C3K2 module is input into the third SEAMHead module. The first SEAMHead module, the second SEAMHead module, and the third SEAMHead module output the identification results of the wearing of safety equipment by each construction worker in the target image.

[0016] Furthermore, the first C3k2_PPA module, the second C3k2_PPA module, the third C3k2_PPA module, and the fourth C3k2_PPA module are four C3k2_PPA modules with the same structure, and each includes a third Conv module, a first Split module, a first PPA module, a second PPA module, a fifth Concat module, and a fourth Conv module connected in series. The output data of the first Split module is also input to the fifth Concat module; the output data of the first PPA module is also input to the fifth Concat module; the input terminal of the third Conv module serves as the input terminal of the C3k2_PPA module, and the output terminal of the fourth Conv module serves as the output terminal of the C3k2_PPA module.

[0017] Furthermore, the first PPA module and the second PPA module are two PPA modules with identical structures, each including a PWConv module, a fifth Conv module, a sixth Conv module, a seventh Conv module, a first Patch-Aware module, a second Patch-Aware module, and an Attention module. The output data of the PWConv module is input to the first Patch-Aware module, the fifth Conv module, and the second Patch-Aware module, respectively. The fifth Conv module, the sixth Conv module, and the seventh Conv module are connected in series. The output data of the first Patch-Aware module, the seventh Conv module, and the second Patch-Aware module are added together and then input to the Attention module. The input end of the PWConv module serves as the input end of the PPA module. The output end of the Attention module serves as the output end of the PPA module.

[0018] Furthermore, the PWConv module is a 1×1 PWConv convolutional layer, and the input data of the PWConv module is F, and , Indicates the height of the input feature map. The width of the input feature map is represented by C, and the number of channels in the input feature map is represented by C. The output data is... ,and , This indicates the number of channels output after PPA processing. The output data of the first Patch-Aware module is local feature data. ,and The output data of the second Patch-Aware module is global features. ,and The output data of the seventh Conv module is the backbone feature. ,and The input data for the Attention module is ,and In the formula, , , , This represents a convolution operation with a 3×3 kernel. This represents a convolution operation with a 7×7 kernel. This represents a convolution operation with a 3×3 kernel.

[0019] Furthermore, the Attention module is a one-dimensional channel attention calculation module, and the output data is... ,and In the formula, , , , , This represents a convolution operation with a 1×1 kernel. GlobalAvgPool() represents a global average pooling operation. () is the Sigmoid activation function that constrains the weights to [0, 1]. MaxPool2D() and AvgPool2D() represent the max pooling and average pooling operations of the 2D convolutional layer, respectively.

[0020] Furthermore, the MPCA module includes the G AvgPool module, Y AvgPool module, X AvgPool module, sixth Concat module, eighth Conv module, ninth Conv module, tenth Conv module, Mean module, Sigmod module, second Split module, third Split module, first weight module, second weight module, first Feature module, second Feature module, and third Feature module. The inputs of the G AvgPool module, Y AvgPool module, and X AvgPool module are used as inputs to the MPCA module to perform C×1×1, C×H×1, and C×1×W convolution operations, respectively, where H, W, and C represent the height, width, and number of channels of the feature map, respectively. The output data of the G AvgPool module is input to the eighth Conv module, Y AvgPool module, and X AvgPool module. The output data of the AvgPool module is input into the sixth Concat module. The output data of the sixth Concat module is input into the ninth Conv module. The output data of the ninth Conv module is input into the tenth Conv module and the second Split module. The output data of the tenth Conv module is input into the Sigmod module. The output data of the Sigmod module is input into the Mean module and the third Split module. The output data of the Mean module and the output data of the eighth Conv module are multiplied and then input into the first Feature module. The output data of the third Split module is input into the first weight module and the second weight module for weight calculation. The output data of the second Split module is multiplied with the output data of the first weight module and the second weight module, and the two multiplication results are input into the second Feature module and the third Feature module, respectively. The output data of the first Feature module, the second Feature module, the third Feature module, and the input data of the MPCA module are multiplied together and then used as the output data of the MPCA module.

[0021] Furthermore, the first, second, and third SEAMHead modules are three structurally identical SEAMHead modules, each including an eleventh Conv module, a first SEAM module, a Conv2D module, a CLSLoss module, a DWConv module, a twelfth Conv module, a second SEAM module, a Conv2D module, and a REG Loss module. The input terminals of the eleventh Conv module and the DWConv module are used together as the input terminals of the SEAMHead module. The eleventh Conv module, the first SEAM module, the Conv2D module, and the CLS Loss module are connected in series, and the CLS Loss module outputs four detection categories. The four detection categories are vest, no-vest, helmet, and no-helmet, representing correctly wearing reflective clothing, not wearing reflective clothing, correctly wearing a helmet, and not wearing a helmet, respectively. The DWConv module, the twelfth Conv module, the second SEAM module, the Conv2D module, and the REG Loss module are connected in series, and the REG Loss module outputs the target bounding box and its geometric information in the target image.

[0022] Furthermore, the first and second SEAM modules are two structurally identical SEAM modules, each including a first CSMM module, a second CSMM module, a third CSMM module, an Average Pooling module, and a Channelexp module. The input terminals of the first, second, and third CSMM modules serve as the input terminals of the SEAM module. The output data of the first, second, and third CSMM modules, along with the input data of the SEAM module, are added together and then input to the Average Pooling module. The output data of the Average Pooling module is then fed into the Channelexp module after passing through two layers of fully connected networks to strengthen the channel connection. The output data of the Channelexp module is multiplied by the input data of the SEAM module and then used as the output data of the SEAM module. The first CSMM module has Patch 6, the second CSMM module has Patch 7, and the third CSMM module has Patch 8.

[0023] Furthermore, the first, second, and third CSMM modules are three CSMM modules with identical structures, each including a Patch Embedding module, a first GELU module, a first BatchNorm module, a Depthwise Convolution module, a second GELU module, a second BatchNorm module, a Pointwise Convolution module, a third GELU module, and a third BatchNorm module. These modules are connected in series. The output data of the first and second BatchNorm modules are added together and then input to the Pointwise Convolution module. The Pointwise Convolution module, the third GELU module, and the third BatchNorm module are then connected in series. The input of the Patch Embedding module serves as the input of the CSMM module, and the output of the third BatchNorm module serves as the output of the CSMM module.

[0024] Compared with existing technologies, the beneficial effects of this invention are as follows: The constructed PHSM-YOLO target detection model can identify the wearing of safety equipment in target images containing construction workers, thereby ensuring the personal safety of construction workers; In the PHSM-YOLO target detection model, P represents the novel C3k2_PPA module, H represents the lightweight downsampling structure HWD module, S represents the novel detection head SEAMHead, and M represents the MPCA attention mechanism. By constructing the novel C3k2_PPA module, the multi-branch parallel structure is used to improve the feature expression ability of multi-scale targets and the fusion ability of local and global information, effectively suppressing background interference while improving the features of small targets; The introduction of the lightweight downsampling module HWD module reduces redundant parameters of the model while retaining more useful information, further improving the model's feature expression ability; The MPCA attention mechanism is introduced at the end of the backbone network to capture contextual information of different scales using a cross-scale attention mechanism, improving the spatial localization ability of small targets; Finally, a separation and enhancement attention module is introduced to construct a new detection head to improve feature extraction ability. Attached Figure Description

[0025] Figure 1 This is a flowchart of the multi-target wearable recognition method of the present invention;

[0026] Figure 2 This is a schematic diagram of the PHSM-YOLO target detection model structure of the present invention;

[0027] Figure 3 This is a structural diagram of the C3k2_PPA of the present invention;

[0028] Figure 4 This is a structural diagram of the MPCA module of the present invention;

[0029] Figure 5 This is a structural diagram of the SEAMHead of the present invention;

[0030] Figure 6 This is a structural diagram of the SEAM module of the present invention;

[0031] Figure 7 This is a structural diagram of the HWD module of the present invention;

[0032] Figure 8 Image samples for three scenes. Detailed Implementation

[0033] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the embodiments described.

[0034] like Figure 1 As shown, the multi-target wearable recognition method based on PHSM-YOLO disclosed in this invention includes the following steps:

[0035] Step 1: Construct a PHSM-YOLO target detection model to identify whether construction workers are wearing safety equipment correctly;

[0036] Step 2: Acquire target images containing construction workers in the construction environment;

[0037] Step 3: Input the target image into the PHSM-YOLO target detection model, and the PHSM-YOLO target detection model will output the recognition results of the safety equipment worn by each construction worker in the target image.

[0038] The constructed PHSM-YOLO target detection model can identify the wearing of safety equipment in target images containing construction workers, thereby ensuring the personal safety of construction workers. In the PHSM-YOLO target detection model, P represents the new C3k2_PPA module, H represents the lightweight downsampling structure HWD module, S represents the new detection head SEAMHead, and M represents the MPCA attention mechanism. By constructing the new C3k2_PPA module, the multi-branch parallel structure is used to improve the feature representation ability of multi-scale targets and the fusion ability of local and global information, effectively suppressing background interference while improving the features of small targets. The introduction of the lightweight downsampling module HWD module reduces redundant parameters of the model while retaining more useful information, further improving the model's feature representation ability. The MPCA attention mechanism is introduced at the end of the backbone network to capture contextual information of different scales using a cross-scale attention mechanism, improving the spatial localization ability of small targets. Finally, a separation and enhancement attention module is introduced to construct a new detection head to improve feature extraction ability.

[0039] like Figure 2 As shown, in step 1, the constructed PHSM-YOLO target detection model includes a backbone network, a neck network, and a detection head;

[0040] The backbone network comprises, in sequence, a first Conv module, a second Conv module, a first C3k2_PPA module, a first HWD module, a second C3k2_PPA module, a second HWD module, a third C3k2_PPA module, a third HWD module, a fourth C3k2_PPA module, an SPPF module, a C2PSA module, and an MPCA module; the input terminal of the first Conv module serves as the input terminal of the backbone network, used to input the target image;

[0041] The neck network includes a first Concat module, a second Concat module, a third Concat module, a fourth Concat module, a first C3K2 module, a second C3K2 module, a third C3K2 module, a fourth C3K2 module, a fourth HWD module, a fifth HWD module, a first Upsample module, and a second Upsample module;

[0042] The output data of the second C3k2_PPA module is also input into the first Concat module. The first Concat module, the first C3K2 module, the fourth HWD module, the second Concat module, the second C3K2 module, the fifth HWD module, the third Concat module, and the third C3K2 module are connected in series to realize the sequential processing of data.

[0043] The output data of the third C3k2_PPA module is also input into the fourth Concat module. The output data of the fourth Concat module is input into the fourth C3K2 module. The output data of the fourth C3K2 module is input into the first Upsample module and the second Concat module respectively. The output data of the first Upsample module is input into the first Concat module.

[0044] The output data of the MPCA module is input into the second Upsample module and the third Concat module respectively, and the output data of the second Upsample module is input into the fourth Concat module.

[0045] The detection head includes a first SEAMHead module, a second SEAMHead module, and a third SEAMHead module. The output data of the first C3K2 module is also input into the first SEAMHead module, the output data of the second C3K2 module is also input into the second SEAMHead module, and the output data of the third C3K2 module is input into the third SEAMHead module. The first SEAMHead module, the second SEAMHead module, and the third SEAMHead module output the identification results of the wearing of safety equipment by each construction worker in the target image.

[0046] The PHSM-YOLO target detection model of the present invention includes a first C3k2_PPA module, a second C3k2_PPA module, a third C3k2_PPA module, and a fourth C3k2_PPA module. Compared with the existing C3k2 module, the C3k2_PPA module can effectively improve the multi-scale target feature extraction capability. By introducing a parallel patch perception module (PPA) to replace the Bottleneck structure in the original C3k2 module, the model's feature extraction capability and detection accuracy for small targets are improved.

[0047] like Figure 3 As shown, the first C3k2_PPA module, the second C3k2_PPA module, the third C3k2_PPA module, and the fourth C3k2_PPA module are four C3k2_PPA modules with the same structure. Each of them includes a third Conv module, a first Split module, a first PPA module, a second PPA module, a fifth Concat module, and a fourth Conv module connected in series. The output data of the first Split module is also input to the fifth Concat module; the output data of the first PPA module is also input to the fifth Concat module; the input terminal of the third Conv module serves as the input terminal of the C3k2_PPA module, and the output terminal of the fourth Conv module serves as the output terminal of the C3k2_PPA module.

[0048] like Figure 3As shown, the first PPA module and the second PPA module are two PPA modules with identical structures, each including a PWConv module, a fifth Conv module, a sixth Conv module, a seventh Conv module, a first Patch-Aware module, a second Patch-Aware module, and an Attention module. The output data of the PWConv module is input to the first Patch-Aware module, the fifth Conv module, and the second Patch-Aware module, respectively. The fifth Conv module, the sixth Conv module, and the seventh Conv module are connected in series. The output data of the first Patch-Aware module, the seventh Conv module, and the second Patch-Aware module are added together and then input to the Attention module. The input terminal of the PWConv module serves as the input terminal of the PPA module, used to receive input data. The output terminal of the Attention module serves as the output terminal of the PPA module, used to output processed data.

[0049] The main advantage of the PPA module used in this invention is its multi-branch feature extraction strategy, which is divided into local, global, and serial convolutional branches. These branches are used to capture detailed information, provide contextual information, and provide basic features, respectively. The local branch is the first Patch-Aware module, the global branch is the second Patch-Aware module, and the serial convolutional branches are the concatenated PWConv module, the fifth Conv module, the sixth Conv module, and the seventh Conv module.

[0050] like Figure 3 As shown, the PWConv module is a 1×1 PWConv convolutional layer, and the input data of the PWConv module is F, and , Indicates the height of the input feature map. The width of the input feature map is represented by C, and the number of channels in the input feature map is represented by C. The output data is... ,and , This indicates the number of channels output after PPA processing. The output data of the first Patch-Aware module is the local feature data output by the local branch. ,and The output data of the second Patch-Aware module is the global features output by the global branch. ,and The output data of the seventh Conv module is the backbone features output by the serial convolution branch. ,and The input data for the Attention module is ,and In the formula, , , , This represents a convolution operation with a 3×3 kernel. This represents a convolution operation with a 7×7 kernel. This represents a convolution operation with a 3×3 kernel.

[0051] In local and global branches, use Unfold to... The data is divided into several continuous blocks, and these blocks are then reshaped and averaged. Then, FFN is used to perform linear computation on the features, where the patch parameter P is the patch size. In the local branch, P=2 focuses on capturing detailed information of small targets to obtain local features; in the global branch, P=4 is used to obtain broader contextual information to obtain global features, as shown in the following formula:

[0052]

[0053] In the formula, L represents the output tensor, which, after expansion, contains all the extracted local patches; the Unfold function represents the expansion / extraction operation; X is the input tensor; P (scalar parameter): Patch Size, i.e., the side length of each patch (e.g., 3x3 or 7x7); P1 and P2 represent the height and width of the window, respectively; the Reshape function represents the reshaping / deformation operation; L flat The vector represents the reshaped feature tensor; B represents the batch size, the number of images processed at one time; H / P represents the number of blocks in the vertical direction and the original height. Divide by block size This gives the number of blocks in the vertical direction; W / P represents the number of blocks in the horizontal direction, based on the original width. Divide by block size This gives the number of blocks in the width direction; P·P represents the number of pixels in a single block; C is the number of channels; m is the output tensor; and Mean represents the mean calculation function.

[0054] Next, the features are processed through two fully connected layers and normalization. This step utilizes a multilayer perceptron (MLP) and normalization to extract a high-level representation of the features, as shown in the following formula:

[0055]

[0056] In the formula, h1, h2, and h3 all represent vectors, equivalent to intermediate variables; the MLP1 function represents the multilayer perceptron (first layer), used for nonlinear transformation; m represents the input feature vector; the Norm function represents the normalization function; and MLP2 is the multilayer perceptron (second layer), used for further nonlinear transformation.

[0057] Next, the activation function softmax transforms the output of FFN into a probability distribution in a spatial dimension and adjusts their weights, as shown in the following formula:

[0058]

[0059] In the formula, attn represents the output attention weight; the Softmax function is the normalization exponential function; the C parameter represents the calculation dimension, usually referring to normalization on the channel dimension or the category dimension; L attn This represents the output tensor, specifically the weighted feature map.

[0060] Finally, a feature selection module performs token and channel feature selection to further enhance local features and fuse global information. The outputs of the three branches are then added together and fed into channel attention and spatial attention for feature enhancement.

[0061] Furthermore, the Attention module is a one-dimensional channel attention calculation module, and the output data is... ,and In the formula, , , , , For each channel weight, The weight for each position, This represents a convolution operation with a 1×1 kernel. GlobalAvgPool() represents a global average pooling operation. () is the Sigmoid activation function that constrains the weights to [0, 1]. MaxPool2D() and AvgPool2D() represent the max pooling and average pooling operations of the 2D convolutional layer, respectively. The weights of each channel are calculated... And two-dimensional spatial attention to calculate the weights at each location Adaptive enhancement of image features to improve focus on key information.

[0062] The existing YOLOv11 uses ordinary convolution for downsampling, which easily leads to the loss of target detail information when handling small target detection tasks such as safety gear in real-world construction scenarios. Furthermore, traditional downsampling methods tend to blur boundary information and lose important targets when processing edge texture details in complex images. To address these issues, this invention introduces an HWD module based on the Hear wavelet downsampling model. The first, second, third, fourth, and fifth HWD modules all employ the same HWD module structure, as shown below. Figure 7As shown, it mainly consists of a lossless feature encoding module and a feature representation learning module. It utilizes Hear wavelet transform to fully preserve detailed information and enhance the receptive field of the feature map, enabling the model to better extract the overall features of the target and improve detection accuracy.

[0063] The lossless feature encoding module primarily utilizes Hear wavelet transform to compress the image spatial resolution without losing image information. It decomposes the feature map with a resolution of H×W×C, then uses a decomposition filter to extract approximate and high-frequency information, achieving multi-scale analysis. Next, it generates approximate low-frequency components and detailed high-frequency components in the horizontal, vertical, and diagonal directions through downsampling, effectively preserving the detail information of small targets. The resolution of each part is... Basis functions of one-dimensional Hear wavelet transform and scaling function The formula is as follows:

[0064]

[0065]

[0066] In the formula, j and k are both orders of the Hear basis functions; It is a scaling function; The scale function is represented by the subscript. Representing scale Translation ; The subscript represents the scaling function after translation. Representing scale Translation ; The normalization coefficient is... Operations represent scaling. This indicates a spatial translation of k units.

[0067] The feature learning module consists of 1×1 convolutional layers, ReLU activation layers, and normalization layers. It is primarily used to remove redundant information and improve the ability of subsequent layers to learn key details. Through these two sub-modules, HWD significantly reduces the computational cost and model size while effectively preserving useful information.

[0068] Due to the complex environment of actual construction sites, severe occlusion between targets, and similarities between targets and the environment, an MPCA attention mechanism is introduced at the end of the backbone network to improve the model's feature extraction capability. This module utilizes height, width, and global channel coordinate information, effectively enhancing the model's feature extraction capability at different scales. Simultaneously, Global Average Pooling (GAP) and more convolutional operations are introduced. Compared to traditional attention mechanisms, MPCA allows the model to learn features more flexibly while considering the entire image content. It preserves key target features while reducing model computation, providing more effective feature representation and detail recovery capabilities for safety wearable device detection. Its structure diagram is shown below. Figure 4As shown, the MPCA module includes the G AvgPool module, Y AvgPool module, X AvgPool module, sixth Concat module, eighth Conv module, ninth Conv module, tenth Conv module, Mean module, Sigmod module, second Split module, third Split module, first weight module, second weight module, first Feature module, second Feature module, and third Feature module. The inputs of the G AvgPool module, Y AvgPool module, and X AvgPool module are used as inputs to the MPCA module to perform C×1×1, C×H×1, and C×1×W convolution operations, respectively, where H, W, and C represent the height, width, and number of channels of the feature map, respectively. The output data of the G AvgPool module is input to the eighth Conv module, Y AvgPool module, and X AvgPool module. The output data of the AvgPool module is input into the sixth Concat module. The output data of the sixth Concat module is input into the ninth Conv module. The output data of the ninth Conv module is input into the tenth Conv module and the second Split module. The output data of the tenth Conv module is input into the Sigmod module. The output data of the Sigmod module is input into the Mean module and the third Split module. The output data of the Mean module and the output data of the eighth Conv module are multiplied and then input into the first Feature module. The output data of the third Split module is input into the first weight module and the second weight module for weight calculation. The output data of the second Split module is multiplied with the output data of the first weight module and the second weight module, and the two multiplication results are input into the second Feature module and the third Feature module, respectively. The output data of the first Feature module, the second Feature module, the third Feature module, and the input data of the MPCA module are multiplied together and then used as the output data of the MPCA module.

[0069] In the detection head of this invention, depthwise separable convolution is fused with SEAM. This effectively solves the problems of target occlusion and small target localization while reducing computational complexity and suppressing redundant information. Its structure is as follows: Figure 5As shown, the first, second, and third SEAMHead modules are three structurally identical SEAMHead modules, each including an eleventh Conv module, a first SEAM module, a Conv2D module, a CLS Loss module, a DWConv module, a twelfth Conv module, a second SEAM module, a Conv2D module, and a REG Loss module. The input terminals of the eleventh Conv module and the DWConv module are used together as the input terminals of the SEAMHead module. The eleventh Conv module, the first SEAM module, the Conv2D module, and the CLS Loss module are connected in series, and the CLS Loss module outputs four detection categories. The four detection categories are vest, no-vest, helmet, and no-helmet, representing correctly wearing reflective clothing, not wearing reflective clothing, correctly wearing a helmet, and not wearing a helmet, respectively. The DWConv module, the twelfth Conv module, the second SEAM module, the Conv2D module, and the REG Loss module are connected in series, and the REG Loss module outputs the target bounding box and its geometric information in the target image.

[0070] YOLO11's original detection head has limitations. It fails to effectively balance the detection performance of small and large targets when handling targets of different scales. Conventional convolutional operation strategies lead to insufficient local recovery ability when facing severely occluded regions. Secondly, the scale of key details of targets varies greatly in complex backgrounds, making accurate localization difficult and increasing the risk of missed detections and false detections. In contrast, the SEAM attention module, through multi-scale design and spatial augmentation techniques, can adapt to different occlusion ratios and pay more attention to features in key regions.

[0071] In the detection and identification of safety equipment at construction sites, the complex environment often leads to obstruction and overlap of targets. To address this issue and improve model recognition accuracy, the SEAM attention module from YOLO-Face is introduced, with the structure as follows: Figure 6As shown, the first and second SEAM modules are two structurally identical SEAM modules, each including a first CSMM module, a second CSMM module, a third CSMM module, an Average Pooling module, and a Channelexp module. The input terminals of the first, second, and third CSMM modules serve as the input terminals of the SEAM module. The output data of the first, second, and third CSMM modules, along with the input data of the SEAM module, are added together and then input to the Average Pooling module. The output data of the Average Pooling module is then fed into the Channelexp module after passing through two layers of fully connected networks to strengthen the channel connection. The output data of the Channelexp module is multiplied by the input data of the SEAM module and then used as the output data of the SEAM module. The first CSMM module has Patch 6, the second CSMM module has Patch 7, and the third CSMM module has Patch 8.

[0072] The SEAM module disclosed in this invention includes three CSMM sub-modules, which extract spatial features using local receptive fields of Patch=6, 7 and 8 respectively.

[0073] Each CSMM module extracts spatial detail information through depthwise separable convolutions, then fuses channel information using 1×1 convolutional kernels, followed by GELU activation and normalization of feature distribution. Finally, features at different scales are enhanced through average pooling and two fully connected layers to strengthen channel connections, thereby improving feature representation and effectively reducing information loss due to occlusion. Figure 6As shown, the first, second, and third CSMM modules are three CSMM modules with identical structures, each including a Patch Embedding module, a first GELU module, a first BatchNorm module, a Depthwise Convolution module, a second GELU module, a second BatchNorm module, a Pointwise Convolution module, a third GELU module, and a third BatchNorm module. The Patch Embedding module, the first GELU module, the first BatchNorm module, the Depthwise Convolution module, the second GELU module, and the second BatchNorm module are connected in series. The output data of the first BatchNorm module and the output data of the second BatchNorm module are added together and then input to the Pointwise Convolution module. The Pointwise Convolution module, the third GELU module, and the third BatchNorm module are connected in series. The input terminal of the Patch Embedding module serves as the input terminal of the CSMM module, and the output terminal of the third BatchNorm module serves as the output terminal of the CSMM module.

[0074] The dataset used in this invention comes from video surveillance footage of a large construction site in Zigong City, Sichuan Province, China. A target detection dataset was constructed by cropping and extracting frames from the video data. This dataset considers close-range (target-camera distance ≤ 5m), medium-range (5m ≤ target-camera distance ≤ 10m), and long-range (target-camera distance ≥ 10m) targets to ensure the diversity and versatility of the safety wearable device dataset. A total of 8437 images were collected and divided into training, testing, and validation sets in an 8:1:1 ratio. Figure 8 Image samples from three scenes are shown.

[0075] This invention uses the labelimg image annotation tool to annotate the detection data. The dataset includes categories of safety equipment such as helmets and reflective vests. The annotated data is saved in YOLO format, and the generated label files correspond one-to-one with the image files. The specific structure of the dataset is shown in Table 1, where vest, no-vest, helmet, and no-helmet represent correctly wearing reflective vests, not wearing reflective vests, correctly wearing helmets, and not wearing helmets, respectively.

[0076] Table 1 shows the composition of the dataset.

[0077] category no-vest vest no-helmet helmet total quantity 1745 2136 7986 5945 17812

[0078] All experiments in this invention were performed on the same server, using Windows 10 as the operating system, Python 3.8 as the programming language, and a CPU (AMD EPYC 7302), GPU (Nvidia Ampere A100), CUDA version 11.3, and PyTorch 1.10.0 as the deep learning framework. In the safety equipment wear detection task, the parameters were set as follows: input image size 640×640 (pixels), SGD optimizer training for 300 iterations, batch size set to 32, and initial learning rate 0.01.

[0079] To verify the performance of the improved detection model, common target detection evaluation metrics were used to evaluate it. Specific evaluation metrics are shown in Table 2. In the evaluation metrics, (↑) indicates that the larger the value, the better the performance, and vice versa (↓) indicates the worse the performance.

[0080] Table 2 shows the target identification evaluation indicators.

[0081] Evaluation indicators definition formula Precision (↑) Accuracy Recall (↑) Recall rate mAP (↑) The average mAP value at a confidence threshold of 0.5 Parameters (↓) Number of parameters in the model —— AP (↑) Average accuracy GFLOPs (↓) computational cost of the model ——

[0082] Where TP is the number of truly existing positive samples detected by the model, FP is the number of negative samples detected as positive samples by the model, FN is the number of positive samples detected as negative samples by the model, and N is the total number of categories.

[0083] To further verify the performance advantages of PHSM-YOLO in safety equipment wear detection, this invention compares it with current mainstream target detection models under otherwise identical conditions, including Faster R-CNN, SSD, CenterNet, and the YOLO series. The aforementioned evaluation metrics are also used for comparative analysis, and the experimental results are shown in Table 3.

[0084] Table 3 compares different detection models.

[0085] Model Precision / % Recall / % mAP / % Parameter quantity / M Computational effort / GFLOPS FasterR-CNN 43.61 62.27 55.65 137.1 370.21 SSD 88.25 46.80 68.18 26.28 62.74 CenterNet 86.56 52.52 67.39 32.67 70.22 YOLOv5 87.42 77.45 83.17 2.74 7.1 YOLOv8 87.53 78.04 84.26 3.0 8.1 YOLOv10 86.62 80.10 84.61 2.66 6.5 YOLOv11 86.7 77.82 84.21 2.58 6.3 PHSM-YOLO 89.13 80.42 86.83 2.31 5.8

[0086] Experiments show that Faster R-CNN, as a two-stage object detection algorithm, performs worse than other one-stage object detection algorithms in terms of model detection accuracy, number of parameters, and computational cost, making it difficult to meet the high efficiency and real-time requirements of safety wearable equipment detection tasks in actual construction scenarios. SSD and CenterNet have similar performance differences, but their detection accuracy is lower than that of the YOLO series models, while their number of parameters and computational cost are much higher, failing to meet the high accuracy and lightweight deployment requirements of detection tasks. Among the YOLO series algorithms, the PHSM-YOLO model constructed in this invention has better detection performance in terms of detection accuracy, number of parameters, and computational cost. Its mAP is improved by 3.66%, 2.22%, 2.57%, and 2.62% compared to v5, v8, v10, and v11, respectively, while the number of model parameters is reduced by 15.7%, 23%, 13.2%, and 10.5%, respectively. Compared to Faster R-CNN, the mAP value is improved by 31.18%, while the number of parameters and model computation are reduced by 98.3% and 98.4%, respectively. Compared to SSD, the mAP is improved by 18.65%, while the number of parameters and computation are reduced by 91.2% and 90.8%, respectively. Compared to CenterNet, the mAP is improved by 19.44%, while the number of parameters and computation are reduced by 92.9% and 91.7%, respectively. Therefore, the PHSM-YOLO model constructed in this invention has better real-time performance and is more advantageous in the task of safe wearable detection.

[0087] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A multi-target wearable recognition method based on PHSM-YOLO, characterized in that, Includes the following steps: Step 1: Construct a PHSM-YOLO target detection model to identify whether construction workers are wearing safety equipment correctly; Step 2: Acquire target images containing construction workers in the construction environment; Step 3: Input the target image into the PHSM-YOLO target detection model, and the PHSM-YOLO target detection model will output the recognition results of the safety equipment worn by each construction worker in the target image.

2. The multi-target wearable recognition method based on PHSM-YOLO according to claim 1, characterized in that, In step 1, the constructed PHSM-YOLO target detection model includes a backbone network, a neck network, and a detection head; The backbone network comprises, in sequence, a first Conv module, a second Conv module, a first C3k2_PPA module, a first HWD module, a second C3k2_PPA module, a second HWD module, a third C3k2_PPA module, a third HWD module, a fourth C3k2_PPA module, an SPPF module, a C2PSA module, and an MPCA module; the input terminal of the first Conv module serves as the input terminal of the backbone network, used to input the target image; The neck network includes a first Concat module, a second Concat module, a third Concat module, a fourth Concat module, a first C3K2 module, a second C3K2 module, a third C3K2 module, a fourth C3K2 module, a fourth HWD module, a fifth HWD module, a first Upsample module, and a second Upsample module; The output data of the second C3k2_PPA module is also input into the first Concat module. The first Concat module, the first C3K2 module, the fourth HWD module, the second Concat module, the second C3K2 module, the fifth HWD module, the third Concat module, and the third C3K2 module are connected in series. The output data of the third C3k2_PPA module is also input into the fourth Concat module. The output data of the fourth Concat module is input into the fourth C3K2 module. The output data of the fourth C3K2 module is input into the first Upsample module and the second Concat module respectively. The output data of the first Upsample module is input into the first Concat module. The output data of the MPCA module is input into the second Upsample module and the third Concat module respectively, and the output data of the second Upsample module is input into the fourth Concat module. The detection head includes a first SEAMHead module, a second SEAMHead module, and a third SEAMHead module. The output data of the first C3K2 module is also input into the first SEAMHead module, the output data of the second C3K2 module is also input into the second SEAMHead module, and the output data of the third C3K2 module is input into the third SEAMHead module. The first SEAMHead module, the second SEAMHead module, and the third SEAMHead module output the identification results of the wearing of safety equipment by each construction worker in the target image.

3. The multi-target wearable recognition method based on PHSM-YOLO according to claim 2, characterized in that, The first, second, third, and fourth C3k2_PPA modules are four C3k2_PPA modules with identical structures. Each module includes a third Conv module, a first Split module, a first PPA module, a second PPA module, a fifth Concat module, and a fourth Conv module connected in series. The output data of the first Split module is also input to the fifth Concat module; the output data of the first PPA module is also input to the fifth Concat module; the input terminal of the third Conv module serves as the input terminal of the C3k2_PPA module, and the output terminal of the fourth Conv module serves as the output terminal of the C3k2_PPA module.

4. The multi-target wearable recognition method based on PHSM-YOLO according to claim 2, characterized in that, The first PPA module and the second PPA module are two PPA modules with the same structure, and each includes a PWConv module, a fifth Conv module, a sixth Conv module, a seventh Conv module, a first Patch-Aware module, a second Patch-Aware module, and an Attention module; The output data of the PWConv module is input to the first Patch-Aware module, the fifth Conv module, and the second Patch-Aware module, respectively. The fifth Conv module, the sixth Conv module, and the seventh Conv module are connected in series. The output data of the first Patch-Aware module, the seventh Conv module, and the second Patch-Aware module are added together and then input to the Attention module. The input of the PWConv module is used as the input of the PPA module. The output of the Attention module is used as the output of the PPA module.

5. The multi-target wearable recognition method based on PHSM-YOLO according to claim 4, characterized in that, The PWConv module is a 1×1 PWConv convolutional layer, and the input data of the PWConv module is F. , Indicates the height of the input feature map. The width of the input feature map is represented by C, and the number of channels in the input feature map is represented by C. The output data is... ,and , This indicates the number of channels output after PPA processing. The output data of the first Patch-Aware module is local feature data. ,and The output data of the second Patch-Aware module is global features. ,and The output data of the seventh Conv module is the backbone feature. ,and The input data for the Attention module is ,and In the formula, , , , This represents a convolution operation with a 3×3 kernel. This represents a convolution operation with a 7×7 kernel. This represents a convolution operation with a 3×3 kernel.

6. The multi-target wearable recognition method based on PHSM-YOLO according to claim 5, characterized in that, The Attention module is a one-dimensional channel attention calculation module, and its output data is... ,and In the formula, , , , , This represents a convolution operation with a 1×1 kernel. GlobalAvgPool() represents a global average pooling operation. () is the Sigmoid activation function that constrains the weights to [0, 1]. MaxPool2D() and AvgPool2D() represent the max pooling and average pooling operations of the 2D convolutional layer, respectively.

7. The multi-target wearable recognition method based on PHSM-YOLO according to claim 2, characterized in that, The MPCA module includes the G AvgPool module, Y AvgPool module, X AvgPool module, sixth Concat module, eighth Conv module, ninth Conv module, tenth Conv module, Mean module, Sigmod module, second Split module, third Split module, first weight module, second weight module, first Feature module, second Feature module, and third Feature module. The inputs of the G AvgPool module, Y AvgPool module, and X AvgPool module are used as inputs to the MPCA module, performing C×1×1, C×H×1, and C×1×W convolution operations respectively, where H, W, and C represent the height, width, and number of channels of the feature map, respectively. The output data of the G AvgPool module is input to the eighth Conv module, Y... The output data from both the AvgPool and XAvgPool modules are input into the sixth Concat module. The output data from the sixth Concat module is input into the ninth Conv module. The output data from the ninth Conv module is input into the tenth Conv module and the second Split module, respectively. The output data from the tenth Conv module is input into the Sigmod module. The output data from the Sigmod module is input into the Mean module and the third Split module, respectively. The output data from the Mean module and the output data from the eighth Conv module are multiplied and then input into the first Feature module. The output data from the third Split module is input into the first weight module and the second weight module for weight calculation. The output data from the second Split module is multiplied by the output data from the first weight module and the second weight module, respectively, and the resulting data is input into the second Feature module and the third Feature module, respectively. The output data from the first Feature module, the second Feature module, the third Feature module, and the input data from the MPCA module are multiplied together and then used as the output data of the MPCA module.

8. The multi-target wearable recognition method based on PHSM-YOLO according to claim 2, characterized in that, The first, second, and third SEAMHead modules are three structurally identical SEAMHead modules, each including an eleventh Conv module, a first SEAM module, a Conv2D module, a CLS Loss module, a DWConv module, a twelfth Conv module, a second SEAM module, a Conv2D module, and a REG Loss module. The input terminals of the eleventh Conv module and the DWConv module are used as the input terminals of the SEAMHead module. The eleventh Conv module, the first SEAM module, the Conv2D module, and the CLS Loss module are connected in series, and the CLS Loss module outputs four detection categories. The four detection categories are vest, no-vest, helmet, and no-helmet, representing correctly wearing reflective clothing, not wearing reflective clothing, correctly wearing a helmet, and not wearing a helmet, respectively. The DWConv module, the twelfth Conv module, the second SEAM module, the Conv2D module, and the REGLoss module are connected in series, and the REG Loss module outputs the target bounding box and its geometric information in the target image.

9. The multi-target wearable recognition method based on PHSM-YOLO according to claim 8, characterized in that, The first and second SEAM modules are two structurally identical SEAM modules, each including a first CSMM module, a second CSMM module, a third CSMM module, an Average Pooling module, and a Channelexp module. The inputs of the first, second, and third CSMM modules serve as the inputs of the SEAM module. The output data of the first, second, and third CSMM modules, along with the input data of the SEAM module, are added together and then input to the Average Pooling module. The output data of the Average Pooling module is then fed into the Channelexp module after passing through two layers of fully connected networks to strengthen the channel connection. The output data of the Channelexp module is multiplied by the input data of the SEAM module and then used as the output data of the SEAM module. The first CSMM module has Patch 6, the second CSMM module has Patch 7, and the third CSMM module has Patch 8.

10. The multi-target wearable recognition method based on PHSM-YOLO according to claim 9, characterized in that, The first, second, and third CSMM modules are three structurally identical CSMM modules, each including a PatchEmbedding module, a first GELU module, a first BatchNorm module, a Depthwise Convolution module, a second GELU module, a second BatchNorm module, a Pointwise Convolution module, a third GELU module, and a third BatchNorm module. These modules are connected in series. The output data of the first and second BatchNorm modules are summed and then input to the Pointwise Convolution module. The Pointwise Convolution module, the third GELU module, and the third BatchNorm module are then connected in series. The input of the PatchEmbedding module serves as the input of the CSMM module, and the output of the third BatchNorm module serves as the output of the CSMM module.

Images