Protective equipment wearing detection method based on deep learning, electronic equipment and product

Abstract

The invention is suitable for the technical field of computer vision, and provides a protective equipment wearing detection method based on deep learning, electronic equipment and a product. The method comprises the following steps: acquiring a to-be-detected image; inputting the to-be-detected image into a personal protection equipment network to obtain a first detection result of the protection equipment and a second detection result of the key part of the human body; determining a protective equipment wearing state of the operator based on the first detection result and the second detection result; a backbone network in the personal protection equipment network comprises at least one layer of space channel distillation enhancement module; and each layer of space channel distillation enhancement module is used for performing cooperative enhancement of the space dimension and the channel dimension on an input first feature map, so that the protection equipment wearing state of an operator can be accurately detected in a complex construction environment.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, and in particular relates to a method, electronic device and product for detecting the wearing of protective equipment based on deep learning. Background Technology

[0002] In urban construction and infrastructure development, safety is always a crucial aspect of project management. Workers entering construction sites are typically required to wear safety helmets, vests, and protective clothing to mitigate safety hazards such as falling objects, vehicle collisions, and environmental risks. However, traditional safety management methods rely primarily on manual inspections and video surveillance playback, which are not only inefficient and costly but also prone to omissions and misjudgments in large-scale construction sites or complex environments.

[0003] With the rapid development of computer vision and deep learning technologies, image recognition-based intelligent safety monitoring is gradually becoming a trend. By automatically identifying the protective equipment (PE) status of workers in images or video streams captured by on-site cameras, real-time, safe, and automated management can be achieved. However, existing methods still face challenges in complex backgrounds, varying lighting conditions, dense crowds, and diverse movement postures during construction, including issues such as low detection accuracy, insufficient model robustness, and difficulty in identifying small targets. This is particularly true for detecting small targets like safety helmets and reflective vests, where false negatives or missed detections are common. Therefore, accurately detecting the PE status of workers in complex construction environments is a pressing technical problem that needs to be solved in the field of construction safety management. Summary of the Invention

[0004] This application provides a deep learning-based method, electronic device, and product for detecting the wearing status of protective equipment by workers in complex construction environments.

[0005] In a first aspect, embodiments of this application provide a deep learning-based method for detecting the wearing of protective equipment, including:

[0006] Acquire an image to be detected, wherein the image to be detected includes at least one key part of the worker's body; The image to be detected is input into the personal protective equipment network to obtain a first detection result for the protective equipment and a second detection result for the key parts of the human body. Based on the first detection result and the second detection result, the protective equipment wearing status of the worker is determined; The personal protective equipment network includes a backbone network and a feature processing network, wherein the backbone network includes at least one layer of spatial channel distillation enhancement module; The spatial channel distillation enhancement module in each layer is used to: perform joint enhancement of the spatial dimension and channel dimension on the input first feature map, and output the second feature map; The feature processing network is used to: determine the first detection result and the second detection result based on the second feature map output by the last layer of the spatial channel distillation enhancement module.

[0007] In this embodiment, by acquiring the image to be detected and inputting it into a personal protective equipment (PPE) network comprising a backbone network and a feature processing network, the spatial channel distillation enhancement module in the backbone network can collaboratively enhance the spatial and channel dimensions of the input feature map. This adaptively highlights the area where the protective equipment is located and the key parts of the worker's body, suppressing feature responses from irrelevant construction backgrounds. This fundamentally improves the discriminative power and anti-interference capability of the feature representation. Based on this, the feature processing network, using the second feature map output by the last layer of the spatial channel distillation enhancement module, can accurately detect workers and collision avoidance equipment in complex construction environments, thereby accurately detecting the worker's PPE wearing status and effectively improving the intelligence and reliability of on-site safety supervision. Furthermore, the PPE network in this solution is built based on deep learning, eliminating the need for manual intervention in the aforementioned PPE wearing detection process and improving the efficiency of PPE wearing status detection.

[0008] In some embodiments of the first aspect, the spatial channel distillation enhancement module includes: a spatial attention submodule, multiple first convolutional layers, a distillation submodule, a fusion submodule, and a channel attention submodule; The spatial attention submodule is used to: enhance the spatial attention of the input first feature map and output a third feature map; The first convolutional layer is used to: perform multi-level local feature extraction, channel expansion and channel restoration on the input third feature map, and output a fourth feature map; The distillation submodule is used to: perform information distillation from the local space to the channel dimension on the input third feature map, and output a fifth feature map; The fusion submodule is used to: fuse the input fourth feature map and the fifth feature map, and output a fused feature map; The channel attention submodule is used to: perform channel attention enhancement on the input fused feature map and output the second feature map.

[0009] In some embodiments of the first aspect, the distillation submodule includes: a second convolutional layer, a local spatial distiller, and a third convolutional layer; The second convolutional layer is used to: perform a projection transformation on the input third feature map and output a sixth feature map; The local spatial distiller is used to: perform local spatial importance distillation on the input sixth feature map, map it to the channel dimension to restore it to a two-dimensional spatial structure, upsample it, and output a reconstructed feature map; The third convolutional layer is used to: adjust the channels of the input reconstructed feature map and output the fifth feature map.

[0010] In some embodiments of the first aspect, the local spatial distiller includes: a splitting unit, a reshaping unit, and a processing unit; The segmentation unit is used to: segment the input sixth feature map into multiple non-overlapping segments in the spatial dimension. Local blocks, Integers greater than zero; The reshaping unit is used to: rearrange the input multiple local blocks to obtain a rearranged feature map, wherein the dimension of the sixth feature map is... The dimension of the rearranged feature map is , Indicates batch size, Indicates the number of input channels. and Indicates spatial resolution; The processing unit is configured to: based on the rearranged feature map, calculate the mean of any rearranged local block in the channel dimension, process it through a feedforward network to obtain the corresponding seventh feature map, process the seventh feature map through a softmax function to obtain a local attention weight matrix, multiply the local attention weight matrix with learnable parameters to obtain an adjusted weight matrix, multiply the adjusted weight matrix element-wise with the seventh feature map to obtain an eighth feature map, restore the eighth feature map to a two-dimensional spatial structure, perform upsampling, and output the reconstructed feature map.

[0011] In some embodiments of the first aspect, the feature processing network includes a Transformer model and a detection head, wherein a spatially aware feedforward network is introduced into the Transformer model to replace the original feedforward network; The spatial perception feedforward network includes a first processing module, a first multi-scale deep convolution module, a stitching module, a second processing module, a second multi-scale deep convolution module, and a third processing module. The first processing module is used to: perform layer normalization on the input feature map and restore it to a two-dimensional spatial structure, and divide it into a ninth feature map and a tenth feature map in the channel dimension; The first multi-scale deep convolution module is used to: extract and fuse features at different scales from the input ninth feature map, and output the eleventh feature map; The splicing module is used to: splice the input tenth feature map and the eleventh feature map to output the twelfth feature map; The second processing module is used to: expand the input twelfth feature map back into sequence form, perform channel expansion, divide it into a thirteenth feature map and a fourteenth feature map in the channel dimension, restore the thirteenth feature map to a two-dimensional spatial structure, and output the fifteenth feature map; The second multi-scale deep convolution module is used to: extract and fuse features at different scales from the input fifteenth feature map, and output the sixteenth feature map; The third processing module is used to: expand the input sixteenth feature map back to sequence form and multiply it element-wise with the fourteenth feature map to obtain the seventeenth feature map; restore the seventeenth feature map to its original dimension and perform a residual connection with the feature map input to the first processing module, wherein the original dimension is the dimension of the feature map input to the first processing module.

[0012] In some embodiments of the first aspect, the first multi-scale deep convolution module is specifically used to: divide the input ninth feature map into multiple eighteenth feature maps in the channel dimension, and concatenate the multiple eighteenth feature maps after performing deep convolution with different kernel sizes to obtain a first concatenated feature map, and then process the first concatenated feature map and the ninth feature map with a residual connection and then process it with the ReLU function to output the eleventh feature map; The second multi-scale deep convolution module is specifically used to: divide the input fifteenth feature map into multiple nineteenth feature maps in the channel dimension, and then concatenate the multiple nineteenth feature maps after performing deep convolution with different kernel sizes to obtain a second concatenated feature map. The second concatenated feature map is then residually connected to the fifteenth feature map and processed by the ReLU function to output the sixteenth feature map.

[0013] In some embodiments of the first aspect, prior to inputting the image to be detected into the personal protective equipment network, the method further includes: Obtain a set of predicted targets and a set of real targets for a sample image; the set of predicted targets includes at least one predicted bounding box for a key part of the human body, and the set of real targets includes at least one real bounding box for a key part of the human body. For any predicted bounding box and any ground truth bounding box, obtain the generalized intersection-over-union ratio between the ground truth bounding box and the predicted bounding box, the probability that the predicted bounding box belongs to the category to which the ground truth bounding box belongs, the length of the diagonal of the envelope of the ground truth bounding box and the predicted bounding box, and the width and height of the envelope. Based on the length of the diagonal, the coordinates of the center point of the predicted bounding box, and the coordinates of the center point of the ground truth bounding box, a center distance constraint between the ground truth bounding box and the predicted bounding box is calculated. The center distance constraint is used to constrain the proximity between the center point of the ground truth bounding box and the center point of the predicted bounding box. Based on the width and height of the predicted bounding box, the width and height of the ground truth bounding box, and the width and height of the envelope bounding box, the shape constraint between the ground truth bounding box and the predicted bounding box is calculated. The shape constraint is used to constrain the similarity between the ground truth bounding box and the predicted bounding box in shape. Based on the probability that the predicted bounding box belongs to the category of the ground truth bounding box, the generalized intersection-union ratio, the center distance constraint, and the shape constraint, the matching cost between the ground truth bounding box and the predicted bounding box is calculated; Based on the probability that the predicted bounding box belongs to the category of the ground truth bounding box, and all matching costs between the predicted target set and the ground truth target set, the loss value of the sample image is calculated, and the personal protective equipment network is trained based on the loss value.

[0014] In some embodiments of the first aspect, the loss value of the sample image includes a regression loss value and a classification loss value; calculating the loss value of the sample image based on the probability that the predicted bounding box belongs to the category of the ground truth bounding box, and all matching costs between the predicted target set and the ground truth target set, includes: For any real bounding box in the set of real targets, determine K candidate bounding boxes that match the real bounding box from the set of predicted targets; the K candidate bounding boxes that match the real bounding box refer to the K predicted bounding boxes with the minimum matching cost with the real bounding box; K is an integer greater than 1; Based on the matching cost between the ground truth bounding box and the K candidate bounding boxes, calculate the weight between the ground truth bounding box and the K candidate bounding boxes; The regression loss value of the sample image is calculated based on the regression loss value and weight between each ground truth bounding box and the K matched candidate bounding boxes. Based on the weights between each ground truth bounding box and each predicted bounding box, and the probability that each predicted bounding box belongs to the category of each ground truth bounding box, the classification loss value of the sample image is calculated. For any ground truth bounding box and the remaining bounding box, the weight between the ground truth bounding box and the remaining bounding box is a preset value. The remaining bounding box is the predicted bounding box that does not belong to the K candidate bounding boxes that match the ground truth bounding box.

[0015] Secondly, embodiments of this application provide a protective equipment wearing detection device based on deep learning, comprising: An image acquisition module is used to acquire an image to be detected, wherein the image to be detected includes at least one key part of the worker's body. The target detection module is used to input the image to be detected into the personal protective equipment network to obtain a first detection result for the protective equipment and a second detection result for the key parts of the human body. The status determination module is used to determine the protective equipment wearing status of the worker based on the first detection result and the second detection result; The personal protective equipment network includes a backbone network and a feature processing network, wherein the backbone network includes at least one layer of spatial channel distillation enhancement module; The spatial channel distillation enhancement module in each layer is used to: perform joint enhancement of the spatial dimension and channel dimension on the input first feature map, and output the second feature map; The feature processing network is used to: determine the first detection result and the second detection result based on the second feature map output by the last layer of the spatial channel distillation enhancement module.

[0016] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device enables the protective equipment wearing detection method as described in any of the first aspects above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a computer, implements the protective equipment wearing detection method as described in any one of the first aspects above.

[0018] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when run, causes the protective equipment wearing detection method as described in any one of the first aspects above to be executed.

[0019] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1This is a schematic diagram of the structure of a personal protective equipment network provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the training method for the personal protective equipment detection network provided in this application embodiment; Figure 3 This is a statistical example diagram of seventeen categories provided in the embodiments of this application; Figure 4 This is a schematic diagram of the process for detecting the wearing of protective equipment based on deep learning, provided in an embodiment of this application. Figure 5 This is a schematic diagram of the structure of the spatial channel distillation enhancement module provided in the embodiments of this application; Figure 6 This is a schematic diagram of the spatial sensing feedforward network provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of the protective equipment wearing detection device based on deep learning provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0023] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0024] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0027] To address the challenges of detecting the wearing of personal protective equipment (PPE) by workers at construction sites, including large differences in target size, complex postures, severe occlusion, and strong background interference, this application proposes an intelligent detection method for PPE based on a Personal Protective Equipment Network (PPE-Net). This method is designed for high-resolution image input scenarios and targets multiple categories of PPE, such as helmets, safety vests, gloves, and goggles. It incorporates innovative designs including enhanced feature extraction, improved feedforward networks, and optimized matching mechanisms to achieve accurate identification and localization of multiple targets in complex construction scenarios.

[0028] In terms of network structure design, the Personal Protective Equipment (PPE) network has undergone systematic innovation in three aspects: feature extraction, feature representation, and target matching. First, a Spatial-Channel Distillation Enhancer (SCDE) module is introduced into the backbone network stage. Through a joint distillation mechanism of spatial saliency and channel attention, the PPE network can adaptively highlight the area where the protective equipment is located and suppress feature responses from irrelevant construction backgrounds, fundamentally improving the discriminative power and anti-interference ability of feature representation. Second, in the encoding and decoding stage of the feature processing network, the feedforward network structure in the traditional Transformer model is improved, and a Spatial-Aware Feedforward Network (SAFFN) is proposed. This network introduces a multi-scale deep convolutional structure and a spatial gating mechanism, enabling dynamic fusion of features between local texture modeling and global semantic understanding. This allows the PPE network to maintain robust detection performance under complex lighting, occlusion, and non-standard pose conditions. Finally, in the sample matching stage, the matching cost is improved, and a soft matching strategy is proposed. This soft matching strategy enhances geometric consistency in the positive and negative sample assignment process by introducing center distance and shape constraints on top of the original matching cost. Furthermore, the soft matching strategy allows a single real target to correspond to multiple predicted boxes with different confidence levels, thereby mitigating the matching instability caused by target overlap and blurred boundaries in complex scenes.

[0029] In summary, the personal protective equipment (PPE) network proposed in this application, while ensuring overall lightweight design and efficient inference, achieves end-to-end optimization from feature extraction to target matching through the collaborative design of a spatial channel distillation enhancement module, a spatial perception feedforward network, and an improved matching mechanism. This significantly improves the detection accuracy and generalization ability of the PPE network in high-resolution construction scenarios. This method can effectively distinguish between wearing and not wearing protective equipment, providing a technical solution with practical engineering value for intelligent construction safety supervision.

[0030] The deep learning-based protective equipment wearing detection method provided in this application can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, desktop computers, servers, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device.

[0031] To illustrate the technical solution of this application, specific embodiments are described below.

[0032] Please see Figure 1 , Figure 1 A schematic diagram of the structure of a personal protective equipment network provided in an embodiment of this application is shown. Figure 1 As shown, the personal protective equipment network includes a backbone network and a feature processing network.

[0033] The aforementioned backbone network includes a Stem module, a four-layer spatial channel distillation enhancement module, and a downsampling module located between each pair of adjacent spatial channel distillation enhancement modules.

[0034] The Stem module is the initial feature extraction module of the backbone network. It is responsible for processing the original input image (e.g., the image to be detected), performing fast spatial downsampling and channel expansion, reducing computational overhead while providing information-dense initial feature maps for subsequent processing.

[0035] The spatial channel distillation enhancement module and downsampling module in the backbone network are alternately connected in series to form a multi-scale feature extraction framework.

[0036] Each spatial channel distillation enhancement module is used to: perform synergistic enhancement of the spatial and channel dimensions of the input first feature map, and output the second feature map.

[0037] The downsampling module is used to downsample the second feature map output by the spatial channel distillation enhancement module above it, and output the downsampled feature map.

[0038] It should be noted that the first feature map input to the first-layer spatial channel distillation enhancement module is the initial feature map, while the first feature maps input to the other three layers of spatial channel distillation enhancement modules are the downsampled feature maps output by the downsampling module of the previous layer.

[0039] Figure 1 In the backbone network, 640×640, 160×160, 80×80, 40×40, and 20×20 represent spatial resolution, and 3, 128, 256, 512, and 1024 represent the number of input channels.

[0040] The feature processing network is used to determine the first detection result and the second detection result based on the second feature map output by the last layer spatial channel distillation enhancement module.

[0041] The feature processing network consists of a Transformer model and a detection head. The Transformer model includes a feature embedding module, an encoder, an output embedding module, and a decoder. The encoder and decoder can have multiple layers, such as... Figure 1 As shown, the feature processing network consists of a six-layer encoder and a six-layer decoder. Figure 1 In this context, q, k, and v represent query, key, and value, respectively, which are the three inputs to the multi-head self-attention layer. The query, key, and value in the encoder come from the output of the backbone network. After passing through the multi-head self-attention layer, the global context modeling of the image can be enhanced, the feature representation can be enhanced, the local limitations of convolution can be overcome, and end-to-end detection can be achieved. Figure 1 The output embedding module is randomly initialized and is learnable. It can include the category and location information of 100 detected targets. Through continuous matching, iteration and information fusion with the encoder output, it eventually learns what kind of output embedding corresponds to what kind of real object.

[0042] In the Transformer model, a spatially aware feedforward network is introduced to replace the original feedforward network, while other modules in the Transformer model (i.e., modules other than the original feedforward network) remain unchanged.

[0043] Figure 1 In the Transformer model, 40 represents the sequence length, 100 represents the number of object queries, B represents the batch size, 256 represents the feature dimension, and PE represents the positional encoding. During the training phase, the batch size is the number of sample images trained in each batch; during the inference phase, the batch size is the number of images to be detected in each batch used for inference.

[0044] The detection head is used to detect protective equipment and key parts of workers' bodies based on the feature map output by the decoder, and obtain the corresponding detection results.

[0045] Figure 1 The detection head outputs a cls: 100×18, which means that the classification prediction output is a matrix with a dimension of (100, 18). For each object query, the detection head outputs an 18-dimensional vector, which represents the confidence of the corresponding prediction box belonging to one of the 17 target categories (including six key human body parts such as people, head, face, ears, hands, and feet, and eleven types of protective equipment such as glasses, medical masks, face shields, earmuffs, gloves, shoes, safety vests, tools, helmets, medical suits, and safety clothing) and a background category. Figure 1 The detection head output reg:100×4 indicates that the regression prediction output is a matrix with a dimension of (100, 4). For each object query, the detection head outputs a 4-dimensional vector (i.e., the four coordinate parameters of the prediction box, which can be the x and y coordinates of the center point and the height and width of the prediction box, or the x and y coordinates of the upper left corner and the lower right corner of the prediction box).

[0046] Please see Figure 2 , Figure 2 The diagram illustrates a flowchart of a training method for a personal protective equipment detection network according to an embodiment of this application. This is an example, not a limitation, and the method is applied to electronic devices. The method includes the following steps: Step 201: Obtain the predicted target set and the real target set of the sample image; the predicted target set includes at least one predicted bounding box of a key human body part, and the real target set includes at least one real bounding box of a key human body part.

[0047] It should be understood that the number of sample images mentioned above can be multiple, so that the personal protective equipment network can learn more complex feature distributions and improve its generalization ability.

[0048] The aforementioned sample images are derived from CSPS17 (Construction Safety Protection Supervision Dataset), a high-quality dataset developed in this embodiment for accurately detecting the protective equipment worn by construction workers. This dataset focuses on the real-world working environment of construction sites, covering various safety protection elements, including safety helmets, safety vests, helmets, earmuffs, gloves, and shoes. The dataset contains nearly 70,000 images, each with a resolution exceeding 4 million pixels, and some reaching 8 million pixels, clearly showing local details of personnel and the wearing status of protective equipment. Data sources include various typical scenarios such as outdoor construction, road maintenance, factory operations, and equipment repair, with diverse shooting angles and lighting conditions, covering various real-world environments such as daytime, nighttime, backlighting, shadows, and complex backgrounds. The category design fully considers the possible combinations of protective equipment and changes in posture that construction workers may exhibit at different stages of work, including both key body parts (head, hands, feet, etc.) and major protective equipment (safety helmets, safety vests, gloves, earmuffs, helmets, shoes, etc.). In terms of annotation, all sample images were annotated with rectangular bounding boxes (i.e., ground truth boxes), and each target object was accompanied by a category label. To ensure annotation accuracy, the annotation process included cross-review by two people and expert review, ensuring that the sample annotation error did not exceed three pixels. This dataset also provides a solid foundation for subsequent personal protective equipment (PPE) wearing detection models to achieve intelligent identification of PPE in complex construction environments, and has significant research and engineering application value. Furthermore, to clearly demonstrate the prevalence and distribution characteristics of each category, this embodiment performed detailed data statistics for each category and presented the results in the form of charts. Figure 3 As shown in the figure, this graph not only reflects the frequency comparison of target occurrences among different categories, but also provides data support for subsequent algorithm optimization, resource allocation, and recognition strategies. Data visualization can effectively guide subsequent optimization work.

[0049] This embodiment uses the PyTorch deep learning framework to build and optimize the proposed personal protective equipment (PPE) network. During the preparation phase, the dataset is collected and cleaned, its labeled data is converted to the format required for model training, and the training and validation sets are divided in an 8:2 ratio to ensure that the PPE network exhibits good generalization ability even on unseen data.

[0050] To enhance the robustness and adaptability of the PPE network, a series of data augmentation techniques were implemented before the dataset was fed into the network. These techniques included, but were not limited to, randomly resizing images, adding random noise, and simulating different weather conditions to represent various scenarios that might be encountered in the real world. These augmented images were then fed into the PPE network, enabling it to learn stably under a wider range of conditions.

[0051] During training, stochastic gradient descent is used to iteratively update the network parameters, seeking the optimal solution by minimizing the loss value. This process essentially involves finding the point with the best performance in the network parameter space. As training progresses, the PPE network gradually learns how to make accurate predictions in various tasks. After training is complete, the final model file is saved, containing not only network structure information but also optimized weight parameters, ensuring direct loading and use during future application deployment. The entire process embodies a series of rigorous steps from data preprocessing to model training and model saving, aiming to build a high-performance deep learning model (i.e., the PPE network) capable of handling complex environments and task requirements.

[0052] Step 202: For any predicted bounding box and any ground truth bounding box, obtain the generalized intersection-over-union ratio between the ground truth bounding box and the predicted bounding box, the probability that the predicted bounding box belongs to the category of the ground truth bounding box, the length of the diagonal of the envelope of the ground truth bounding box and the predicted bounding box, and the width and height of the envelope.

[0053] The aforementioned envelope can refer to the smallest outermost matrix containing both the ground truth box and the predicted box.

[0054] In the end-to-end detection framework based on the Transformer model, the positive and negative sample matching for object detection relies on the Hungarian algorithm to achieve a one-to-one assignment, that is, in the predicted target set of each sample image. With the real target set Between these, we find an optimal match, and the cost function of the match can be expressed as follows:

[0055] in, Indicates the first The first real frame and the first The cost of matching between prediction boxes; Indicates the first Each prediction box belongs to the category The probability, Indicates the first The category to which each real frame belongs; Indicates the first The first real frame and the first Manhattan distance between prediction boxes; Indicates the first The first real frame and the first Generalized intersection-union ratio among prediction boxes; , , These represent the weights for each item. N represents the number of predicted boxes, and M represents the number of ground truth boxes.

[0056] While this matching method has some effectiveness in detection tasks, it also has some shortcomings. For example, the Manhattan distance is summed separately in the x and y directions, failing to reflect the true spatial distance of the center point deviation; the matching process ignores the shape similarity between the predicted and ground truth bounding boxes, relying solely on position and intersection-over-union (IoU) constraints, without considering scale features such as height and width, which can easily lead to mismatches of targets with different shapes; the one-to-one matching also limits the learning efficiency of the PPE network, as each ground truth bounding box has only one matched predicted bounding box, resulting in too few positive samples and slow training convergence. To more accurately select positive samples, this embodiment improves the traditional matching method from three aspects: center distance constraints, shape constraints, and loss values, thereby providing support for subsequent loss value calculation and accelerating the convergence speed of the PPE network.

[0057] Step 203: Based on the length of the diagonal, the coordinates of the center point of the predicted box, and the coordinates of the center point of the ground truth box, calculate the center distance constraint between the ground truth box and the predicted box. The center distance constraint is used to constrain the proximity between the center point of the ground truth box and the center point of the predicted box.

[0058] In this embodiment, the center distance between the ground truth box and the predicted box can be calculated based on the length of the diagonal, the coordinates of the center point of the predicted box, and the coordinates of the center point of the ground truth box, and the center distance constraint can be calculated based on this center distance.

[0059] center distance The calculation formula is as follows:

[0060] Center distance constraint The calculation formula is as follows:

[0061] in, Represents a real box With prediction box The length of the diagonal of the envelope; Represents a real box The coordinates of the center point; Indicates the coordinates of the center point of the prediction box; This represents the attenuation scale of the control center distance constraint, which can be set according to actual needs or empirical values.

[0062] In this embodiment, by introducing a center distance constraint, the matching process no longer relies solely on the L1 distance difference. Instead, it uses the L2 norm to measure the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, thus more accurately reflecting the spatial offset relationship between targets. Compared to the L1 distance, the L2 constraint is geometrically continuous and more sensitive to center offset, enabling more effective matching in space.

[0063] Step 204: Based on the width and height of the predicted bounding box, the width and height of the ground truth bounding box, and the width and height of the envelope bounding box, calculate the shape constraint between the ground truth bounding box and the predicted bounding box. The shape constraint is used to constrain the similarity between the ground truth bounding box and the predicted bounding box in terms of shape.

[0064] A first value can be calculated based on the width of the predicted bounding box, the width of the ground truth bounding box, and the width of the envelope bounding box; a second value can be calculated based on the height of the predicted bounding box, the height of the ground truth bounding box, and the height of the envelope bounding box; and then the shape constraint can be calculated based on the first and second values.

[0065] First value Second value The calculation formula is as follows:

[0066] Shape constraints The calculation formula is as follows:

[0067] in, and Represents a real box With prediction box The width and height of the envelope; and Represents a real box Width and height; and Represents the prediction box Width and height; This indicates the attenuation scale for controlling shape constraints, which can be set according to actual needs or empirical values.

[0068] In this embodiment, shape constraints explicitly incorporate scale similarity factors of the target's width and height into the matching cost, compensating for the inadequacy of geometric shape difference representation in the matching loss. By jointly constraining the height and width of the predicted and ground truth boxes, the personal protective equipment network can perceive the target's morphological information during the matching stage, enabling more accurate positive sample assignment for objects such as elongated shapes and small-scale targets. This helps reduce matching bias caused by geometric structural differences and enhances the personal protective equipment network's ability to discriminate targets of different scales.

[0069] Step 205: Calculate the matching cost between the ground truth box and the predicted box based on the probability that the predicted box belongs to the category of the ground truth box, the generalized intersection-union ratio, the center distance constraint, and the shape constraint.

[0070] By adding center distance constraints and shape constraints, the improved matching cost can be expressed as:

[0071] in, Indicates the weight of the shape constraint.

[0072] Step 206: Based on the probability that the predicted bounding box belongs to the category of the true bounding box, and all matching costs between the predicted target set and the true target set, calculate the loss value of the sample image, and train the personal protective equipment network based on the loss value.

[0073] In this embodiment, after training the personal protective equipment (PPE) network, the trained PPE network can be used to detect whether workers at the construction site are wearing PPE in the images to be detected. It should be understood that the trained PPE network can also be referred to as a PPE model.

[0074] In some embodiments of this application, the loss value of the sample image includes a regression loss value and a classification loss value; the loss value of the sample image is calculated based on the probability that the predicted bounding box belongs to the category of the true bounding box, and all matching costs between the predicted target set and the true target set, including: For any ground truth bounding box in the set of ground truth objects, determine K candidate bounding boxes that match the ground truth bounding box from the set of predicted objects; the K candidate bounding boxes that match the ground truth bounding box are the K predicted bounding boxes with the minimum matching cost with the ground truth bounding box; K is an integer greater than 1. Calculate the weights between the ground truth bounding box and the K candidate bounding boxes based on the matching cost between the ground truth bounding box and the K candidate bounding boxes; The regression loss value of the sample image is calculated based on the regression loss value and weight between each ground truth bounding box and the K matched candidate bounding boxes. Based on the weights between each ground truth bounding box and each predicted bounding box, and the probability that each predicted bounding box belongs to the category of each ground truth bounding box, the classification loss value of the sample image is calculated. For any ground truth bounding box and the remaining bounding box, the weights between the ground truth bounding box and the remaining bounding box are preset values, and the remaining bounding boxes are the predicted bounding boxes that do not belong to the K candidate boxes that match the ground truth bounding boxes.

[0075] After obtaining the complete matching cost, instead of selecting a single optimal solution, for each ground truth box, the cost is calculated across all predictions. Find the smallest set of K indices The original positive sample pairs are expanded, and soft labels are assigned to the candidate boxes in the set. This means that the matching cost between the ground truth box and each candidate box is divided by the ratio of the total matching cost between the ground truth box and the K candidate boxes. This ratio is the weight between the ground truth box and the corresponding candidate box, thereby generating a normalized weight based on the matching cost.

[0076] Weights between the ground truth bounding box and the matched candidate bounding box The calculation formula is as follows:

[0077] in, This represents the matching cost between the ground truth bounding box and the candidate bounding box. Indicates the index of the candidate box.

[0078] Regression loss value of sample images The calculation formula is as follows:

[0079] in, Indicates adoption The regression loss function calculates the regression loss value between the true bounding box and the candidate bounding box.

[0080] The formula for calculating the classification loss value of the sample image is as follows:

[0081] in, The sign function indicates whether the predicted bounding box is a positive sample. If the predicted bounding box is a positive sample (i.e., a candidate bounding box), the corresponding sign function value is 1, and the corresponding weight is... for If the predicted bounding box is a negative sample, the corresponding sign function value is 0, and the corresponding weight is 1. Represents the prediction box Category The probability of.

[0082] In this embodiment, soft matching, based on traditional one-to-one Hungarian matching, introduces a multi-positive-sample soft matching mechanism, allowing each ground truth bounding box to simultaneously correspond to K low-cost candidate bounding boxes, with weights dynamically allocated according to the matching cost. This mechanism not only enriches the number of positive samples but also improves the density and stability of gradient propagation, accelerating the convergence of the PPE network. Through the above training, the PPE network can also fully utilize the learning signals of similar predicted boxes, avoiding the training jitter problem caused by over-reliance on a single optimal match, thereby achieving a smoother optimization process and stronger generalization ability.

[0083] Please see Figure 4 , Figure 4This illustration shows a flowchart of a deep learning-based protective equipment wearing detection method provided in an embodiment of this application, that is, through methods such as... Figure 2 After training the personal protective equipment (PPE) detection network in the illustrated embodiment, the method implements PPE wearing detection based on the trained PPE detection network. As an example and not a limitation, the method includes the following steps: Step 401: Obtain the image to be detected.

[0084] The image to be detected includes at least one key part of the worker's body and may also include protective equipment to detect whether the worker is wearing protective equipment on that key part of their body. The key part of the body can refer to a part of the body where protective equipment is regularly worn. The number of workers in the image to be detected can be one or more (i.e., at least two), and this application does not limit this. The number of protective equipment pieces in the image to be detected can be zero, one, or at least two. When the number of protective equipment pieces is zero, it can be determined that the worker is not wearing protective equipment. When the number of protective equipment pieces is multiple, these multiple protective equipment pieces can be of the same type or different types (e.g., they can include multiple types of protective equipment such as safety helmets, safety vests, and helmets).

[0085] As an example and not a limitation, the image to be detected can be a normalized three-channel RGB image with a resolution of 640×640. The RGB image can include workers and protective equipment such as safety helmets, safety vests, and helmets.

[0086] As an example and not a limitation, at least one camera is installed at the construction site, which can capture images of workers, identify the captured images of workers as images to be detected, and transmit them to electronic devices. The electronic devices can detect the wearing of protective equipment in the images to be detected through steps 402 and 403.

[0087] Step 402: Input the image to be detected into the personal protective equipment network to obtain the first detection result of the protective equipment and the second detection result of the key parts of the human body.

[0088] The first detection result indicates whether protective equipment exists in the image to be detected, and if so, includes the type of protective equipment and a predicted bounding box. The second detection result indicates whether critical parts of a worker's body exist in the image to be detected, and if so, includes the type of the critical parts and a predicted bounding box.

[0089] In step 402, the personal protective equipment (PPE) network is a pre-trained PPE network. After the electronic device inputs the image to be detected into the pre-trained PPE network, the PPE network can output the detection results for the key human body parts and protective equipment of the worker in the image, achieving high-precision positioning and category determination of the key human body parts and protective equipment.

[0090] The aforementioned personal protective equipment network includes: a backbone network and a feature processing network, wherein the backbone network includes at least one layer of spatial channel distillation enhancement module; Each spatial channel distillation enhancement module is used to: perform joint enhancement of the spatial dimension and channel dimension of the input first feature map, and output the second feature map; The feature processing network is used to determine the first detection result and the second detection result based on the second feature map output by the last layer spatial channel distillation enhancement module.

[0091] In this embodiment, the spatial channel distillation enhancement module can adaptively focus on the most distinctive local details of key parts of the human body and protective equipment through the synergistic enhancement of the spatial dimension and the channel dimension.

[0092] When the spatial channel distillation enhancement module has at least two layers, a downsampling module can be set between every two adjacent spatial channel distillation enhancement modules. Optionally, such as... Figure 1 As shown, the backbone network may include four layers of spatial channel distillation enhancement modules, with a downsampling module between each pair of adjacent spatial channel distillation enhancement modules.

[0093] In this embodiment, the downsampling module can progressively expand the receptive field, enabling the backbone network to understand the global context. This is crucial for identifying protective equipment in occluded or dense scenes. By alternately connecting the spatial channel distillation enhancement module and the downsampling module, an efficient multi-degree feature extraction framework can be constructed. This allows the backbone network to simultaneously possess macro-contextual understanding and local detail discrimination capabilities. Consequently, regardless of the size or distance of the protective equipment and key human body parts in the image to be detected, the backbone network can always possess feature representations at a certain level or several levels of feature maps that can both "see clearly" their details (spatial focusing) and "understand" their meaning (channel enhancement). This allows the feature processing network to achieve high-precision classification and localization of protective equipment and key human body parts based on these carefully selected and enhanced final features.

[0094] Step 403: Based on the first and second test results, determine the protective equipment wearing status of the workers.

[0095] The protective equipment wearing status indicator shows whether the workers are wearing protective equipment on key parts of their bodies.

[0096] In this embodiment, if the first detection result indicates that at least one protective device exists in the image to be detected, and the second detection result indicates that at least one critical human body part exists in the image to be detected, then for any protective device and any critical human body part, it can be detected whether the prediction frame of the protective device is located within the prediction frame of the critical human body part. If the prediction frame of the protective device is located within the prediction frame of the critical human body part, it is determined that the critical human body part is wearing the protective device; if the prediction frame of the protective device is not located within the prediction frame of the critical human body part, it is determined that the critical human body part is not wearing the protective device. It should be understood that the protective device wearing status of the worker can also be determined based on the first and second detection results in other ways, and this application does not limit this.

[0097] In this embodiment, by acquiring the image to be detected and inputting it into a personal protective equipment (PPE) network comprising a backbone network and a feature processing network, the spatial channel distillation enhancement module in the backbone network can collaboratively enhance the spatial and channel dimensions of the input feature map. This adaptively highlights the area where the protective equipment is located and the key parts of the worker's body, suppressing feature responses from irrelevant construction backgrounds. This fundamentally improves the discriminative power and anti-interference capability of the feature representation. Based on this, the feature processing network, using the second feature map output by the last layer of the spatial channel distillation enhancement module, can accurately detect workers and collision avoidance equipment in complex construction environments, thereby accurately detecting the worker's PPE wearing status and effectively improving the intelligence and reliability of on-site safety supervision. Furthermore, the PPE network in this solution is built based on deep learning, eliminating the need for manual intervention in the aforementioned PPE wearing detection process and improving the efficiency of PPE wearing status detection.

[0098] In some embodiments of this application, the spatial channel distillation enhancement module includes: a spatial attention submodule, multiple first convolutional layers, a distillation submodule, a fusion submodule, and a channel attention submodule; The spatial attention submodule is used to: enhance the spatial attention of the input first feature map and output the third feature map; The first convolutional layer is used to perform multi-level local feature extraction, channel expansion, and channel restoration on the input third feature map, and output the fourth feature map. The distillation submodule is used to: distill information from the local space to the channel dimension from the input third feature map, and output the fifth feature map; The fusion submodule is used to fuse the input fourth and fifth feature maps and output a fused feature map. The channel attention submodule is used to enhance the input fused feature map with channel attention and output a second feature map.

[0099] The first convolutional layer can be a 3×3 kernel layer used to generate high-level semantic features related to protective equipment and key parts of the human body. Each first convolutional layer may be followed by batch normalization and activation functions.

[0100] In this embodiment, firstly, the spatial attention submodule enhances the spatial attention of the input first feature map, adaptively suppressing interference from complex construction backgrounds and initially highlighting potential target areas, thus completing the first enhancement of the spatial dimension. Subsequently, the third feature map undergoes deep processing along two parallel paths. One path uses multiple first convolutional layers for progressive local feature extraction, channel expansion, and channel recovery, producing highly semantic convolutional output; the other path uses a distillation submodule to distill information from the local spatial dimension to the channel dimension of the third feature map, extracting the most discriminative compact channel features. The fusion submodule deeply fuses the feature maps output from the two paths, achieving the first synergy between spatial details and channel semantics. Finally, the channel attention submodule enhances the fused feature map with channel attention, further strengthening feature channels related to protective equipment categories and key human body parts, suppressing irrelevant channels, thus completing the ultimate enhancement of the channel dimension.

[0101] In some embodiments, the spatial attention submodule may include a first max pooling layer, a first average pooling layer, a concatenation module, a fourth convolutional layer, a sigmoid function, a multiplication module, etc. After the first feature map is input into the spatial attention submodule, the first average pooling layer and the first max pooling layer calculate the element-wise average and maximum values ​​for each channel dimension, respectively, to obtain two single-channel feature maps. These are then concatenated along the channel dimension by the concatenation module to form a feature map with dimension 1. tensor, This indicates the batch size, and 2 indicates the number of input channels. and The spatial resolution is represented by a 3×3 two-dimensional convolutional layer and mapped by the sigmoid function to generate a spatial attention weight matrix. This matrix is ​​then multiplied element-wise with the first feature map by a multiplication module to obtain the third feature map.

[0102] The aforementioned channel attention submodule may include a second average pooling layer, a second max pooling layer, a fusion submodule, a multilayer perceptron (MLP), and a multiplication module. The fusion submodule may be a residual connection. The MLP may include two fully connected layers and a sigmoid function. After the fused feature map is input into the channel attention submodule, adaptive max pooling and average pooling are performed by the second max pooling layer and the second average pooling layer, respectively, to obtain channel descriptions. These descriptions are then mapped by the two fully connected layers in the MLP and the channel attention weight matrix is ​​obtained through the sigmoid function. After being expanded to the same spatial size as the fused feature map, the matrix is ​​multiplied channel-by-channel by the multiplication module to achieve channel recalibration and enhance the spatial attention of the fused feature map.

[0103] In some embodiments of this application, the distillation submodule includes: a second convolutional layer, a local spatial distiller, and a third convolutional layer; The second convolutional layer is used to: perform a projection transformation on the input third feature map and output the sixth feature map; The local spatial distiller is used to: perform local spatial importance distillation on the input sixth feature map, map it to the channel dimension to restore the two-dimensional spatial structure, and then upsample it to output a reconstructed feature map; The third convolutional layer is used to adjust the channels of the input reconstructed feature map and output the fifth feature map.

[0104] The second and third convolutional layers can be two-dimensional convolutional layers with a kernel size of 1×1.

[0105] In this embodiment, the distillation submodule efficiently achieves directional distillation and refinement from local spatial information to high-order channel semantics through a three-stage pipeline of projection, distillation, and reconstruction. First, the projection transformation performed by the second convolutional layer maps features to an intermediate representation space or feature subspace more suitable for importance assessment, preparing data for subsequent distillation steps. Second, the local spatial distiller intelligently identifies and retains the most critical local region features for distinguishing protective equipment from key human body parts by performing local spatial importance distillation on the sixth feature map, while filtering out background noise. Subsequently, after mapping it to the channel dimension to restore the two-dimensional spatial result and performing upsampling, the selected key spatial location information is transformed into a reconstructed feature map emphasizing the channel semantic importance weights. Finally, the third convolutional layer adjusts the channels of the reconstructed feature map, ensuring that the channel weight information produced by distillation has appropriate dimension and intensity, enabling effective fusion with the rich detailed features output from another path (the fourth feature map output by the multi-layer first convolutional layer).

[0106] In some embodiments of this application, there are a segmentation unit, a reshaping unit, and a processing unit; The segmentation unit is used to: segment the input sixth feature map into multiple non-overlapping segments in the spatial dimension. Local blocks, Integers greater than zero; The reshaping unit is used to rearrange multiple local blocks of the input to obtain rearranged feature maps. The sixth feature map has a dimension of . The dimension of the rearranged feature map is , Indicates batch size, Indicates the number of input channels. and Indicates spatial resolution; The processing unit is used to: calculate the mean of any rearranged local block in the input along the channel dimension based on the rearranged feature map, and then process it through a feedforward network to obtain the corresponding seventh feature map. The seventh feature map is then processed by the softmax function to obtain the local attention weight matrix. The local attention weight matrix is ​​multiplied by the learnable parameters to obtain the adjusted weight matrix. The adjusted weight matrix is ​​then multiplied element-wise with the seventh feature map to obtain the eighth feature map. The eighth feature map is then restored to a two-dimensional spatial structure and then upsampled to output the reconstructed feature map.

[0107] Optionally, the spatial resolution of the local blocks can be set according to actual needs or empirical values, and the sixth feature map can be segmented based on this spatial resolution. As an example, and not a limitation, the spatial resolution of the local blocks is... , For example, integers greater than zero. The value is 2.

[0108] in, This represents the length of the flattened spatial resolution of each local block.

[0109] In this embodiment, by rearranging multiple local blocks of the input through the reshaping unit, the two-dimensional spatial structure of the sixth feature map (the height and width of the sixth feature map) can be converted into a sequence structure based on local blocks, so as to facilitate the averaging operation on the channel dimension.

[0110] Optionally, the eighth feature map can be restored to a two-dimensional spatial structure by reshaping the unit.

[0111] In this embodiment, the segmentation unit divides the sixth feature map into non-overlapping local blocks in the spatial dimension, enabling subsequent evaluation to focus on each local region of the image with fine granularity. This is crucial for capturing details of small protective equipment and key parts of the human body. The rearrangement operation establishes a sequential representation for these local blocks that is easy to process in parallel. The processing unit averages each local block in the channel dimension, essentially extracting the "channel consensus" or average activation intensity of that local region to obtain a preliminary saliency scalar. Subsequently, through feedforward processing and normalization using the softmax function, this process transforms the preliminary saliency into a comparable, competitive local attention weight matrix. This weight directly "represents local spatial importance," that is, adaptively quantifies the relative contribution of each local block to the final task. Then, learnable parameters are introduced to fine-tune the local attention weight matrix, enhancing the flexibility and expressive power of the processing unit. Finally, the adjusted weight matrix is ​​multiplied element-wise with the seventh feature map. This is equivalent to performing an adaptive spatial recalibration of the local block features using the distilled spatial importance weight matrix, which enhances important local blocks and weakens less important ones. Next, after restoring the spatial shape of these recalibrated and dispersed eighth feature maps, upsampling is performed. This allows the recalibrated features of all local blocks to be reassembled into a complete feature map aligned with the original input spatial dimensions, i.e., a reconstructed feature map.

[0112] In this embodiment, the spatial channel distillation enhancement module implements a complete chain from spatial attention-guided positional emphasis, learnable local spatial distillation, to deep convolutional semantic extraction and channel recalibration. Specifically, while maintaining the input spatial resolution, it compresses local spatial details and injects them into the channel representation, enhancing the responsiveness to small-scale and detailed targets, and providing the encoder of the subsequent feature processing network with more discriminative and small-target-friendly feature representations.

[0113] In some embodiments, before restoring the eighth feature map to a two-dimensional spatial structure, the eighth feature map can be first layer-normalized to obtain a normalized feature map. Then, a matrix multiplication module is used to perform matrix multiplication between the normalized feature map and the eighth feature map to obtain the processed feature map. like Figure 5 The diagram shown is a structural schematic of the spatial channel distillation enhancement module provided in the embodiment of this application. Figure 5In this diagram, Mean represents the first average pooling layer, Max represents the first max pooling layer, dim=1 represents the channel dimension, LSD represents the local spatial distiller, Conv2d represents a 2D convolutional layer, k=3 represents a 3×3 convolutional kernel, k=1 represents a 1×1 convolutional kernel, AvgPool represents the second average pooling layer, MaxPool represents the second max pooling layer, add represents a residual connection, Unfold represents a splitting unit, Reshape represents a reshaping unit, FFN represents a feedforward network, and Upsample represents upsampling. This represents ×. As an example, not a limitation, let the first feature map be denoted as . Based on the structural schematic diagram of the space channel distillation enhancement module shown in Figure 5, the main data processing flow of the forward propagation process of the space channel distillation enhancement module can be represented as follows: The spatial attention submodule's processing of the first feature map can be represented as follows: By performing average pooling and max pooling on the channel dimension of the first feature map, the resulting two single-channel feature maps can be represented as follows:

[0114] in, This represents the single-channel feature map corresponding to average pooling. This represents the single-channel feature map corresponding to max pooling. This represents the element in the i-th row and j-th column of the c-th channel of the first feature map.

[0115] The formula for calculating the spatial attention weight matrix can be expressed as follows:

[0116] in, Represents the spatial attention weight matrix. This represents the sigmoid function. This represents a two-dimensional convolutional layer with a kernel size of 3×3. This indicates the splicing module.

[0117] The formula for calculating the third feature map can be expressed as follows:

[0118] in, Represents the third feature map. This indicates the multiplication module.

[0119] The local distillation branch (i.e., the distillation submodule) processes the third feature map as follows: The second convolutional layer performs a projection transformation on the third feature map, outputting the sixth feature map. The formula for calculating the sixth feature map can be expressed as follows:

[0120] in, Represents the sixth feature map. This represents a two-dimensional convolutional layer with a kernel size of 1×1.

[0121] The sixth feature map is spatially segmented into non-overlapping parts. The local blocks, in total, have A local block. For this The local blocks are rearranged, and the formula for calculating the feature map after rearrangement can be expressed as follows:

[0122] in, This represents the rearranged feature map. Indicates the segmentation unit, This indicates a remodeling unit.

[0123] Since the rearranged feature map is composed of local blocks, for ease of description, all local blocks can be treated as a whole. The averaging process of the local block containing any element at any position in the rearranged feature map can be described by the element at that position in the channel dimension. The formula for calculating the average of the element at any position in the rearranged feature map in the channel dimension can be expressed as follows:

[0124] in, Indicates the first element in the rearranged feature map. Line 1 The mean of the elements in the column along the channel dimension; Indicates the first element in the rearranged feature map. Line 1 The elements of the column.

[0125] The seventh feature map is obtained through the feedforward network after averaging. The calculation formula is as follows:

[0126] in, and This represents two fully connected layers in a feedforward network; This represents layer normalization applied between the two fully connected layers. The dimensionality changes of the feature map before and after processing by the above two fully connected layers are as follows:

[0127]

[0128] in, and This represents the adjusted number of input channels in the fully connected layer of the feedforward network.

[0129] Adjusted weight matrix The calculation formula is as follows:

[0130] in, Learnable parameters: The above eighth feature map The calculation formula can be expressed as follows:

[0131] Reconstructing feature maps The calculation formula can be expressed as follows:

[0132] The third convolutional layer performs channel adjustment on the reconstructed feature map, outputting the fifth feature map. The calculation formula can be expressed as follows:

[0133] The implementation process of a convolutional branch (i.e., multiple first convolutional layers) can be represented as follows:

[0134]

[0135]

[0136] in, This represents the feature map output by the first convolutional layer. This represents the feature map output by the first convolutional layer of the second layer; This represents the feature map output by the first convolutional layer of the third layer (i.e., the fourth feature map). This indicates batch normalization.

[0137] The fused feature map output by the fusion submodule The calculation formula can be expressed as follows:

[0138] The second feature map output by the channel attention submodule The calculation formula can be expressed as follows:

[0139]

[0140]

[0141] in, and These represent the feature maps output by the second average pooling layer and the second max pooling layer, respectively. This represents the channel attention weight matrix.

[0142] In the task of detecting personal protective equipment (PPE) for workers, PPE networks need to simultaneously identify key parts of the human body and various protective equipment worn by them, such as helmets, safety vests, earmuffs, and gloves. These targets exhibit significant scale and semantic differences in the same image: protective equipment is usually small in size, scattered in location, and easily occluded, while large targets such as the human body and clothing contain continuous textures and complex postures. Traditional convolutional features often have a trade-off between multi-scale structural representation and fine-grained semantic preservation, making it difficult to simultaneously ensure the accuracy of small target detection and the structural integrity of large targets. The spatial channel distillation enhancement module achieves synergistic enhancement of global and local features by combining spatial distillation and channel recalibration, while maintaining lightweight design: its spatial distillation branch extracts high-resolution texture features of local key regions (such as the head and hands), effectively amplifying the response of small targets in the feature map; while the convolutional branch captures global contextual relationships, maintaining spatial consistency between human posture and protective equipment, thereby obtaining more discriminative semantic representations in complex scenes.

[0143] Furthermore, the channel attention mechanism in the spatial channel distillation enhancement module adaptively adjusts the weights of different semantic channels based on the statistical information of the input features, enabling the personal protective equipment (PPE) network to dynamically focus on key PPE features and key human body features while suppressing background interference. In operational scenarios, image backgrounds often contain a large amount of interfering information (such as machinery, ground textures, and piles of debris). The channel recalibration mechanism effectively enhances the PPE network's responsiveness to target areas, significantly improving the recall rate and localization accuracy of PPE wearing detection. This provides an efficient and reliable feature enhancement scheme for multi-category safety protection detection in high-resolution images.

[0144] In some embodiments of this application, the Transformer model and the detection head are used, and a spatially aware feedforward network is introduced into the Transformer model to replace the original feedforward network.

[0145] The spatial awareness feedforward network includes a first processing module, a first multi-scale deep convolution module, a stitching module, a second processing module, a second multi-scale deep convolution module, and a third processing module; The first processing module is used to: perform layer normalization on the input feature map and restore it to a two-dimensional spatial structure, and divide it into the ninth feature map and the tenth feature map in the channel dimension; The first multi-scale deep convolution module is used to: extract and fuse features at different scales from the input ninth feature map, and output the eleventh feature map; The concatenation module is used to concatenate the input tenth and eleventh feature maps and output the twelfth feature map. The second processing module is used to: expand the input twelfth feature map back into sequence form, perform channel expansion, divide it into thirteenth and fourteenth feature maps in the channel dimension, restore the thirteenth feature map to a two-dimensional spatial structure, and output the fifteenth feature map; The second multi-scale deep convolution module is used to: extract and fuse features at different scales from the input fifteenth feature map, and output the sixteenth feature map; The third processing module is used to: expand the input sixteenth feature map back to sequence form and multiply it element-wise with the fourteenth feature map to obtain the seventeenth feature map; restore the seventeenth feature map to its original dimension and perform a residual connection with the feature map input to the first processing module, where the original dimension is the dimension of the feature map input to the first processing module.

[0146] Optionally, after expanding the input twelfth feature map back into sequence form, channel expansion can be performed, specifically expanding the channels to a preset number. Optionally, the preset number can be set according to actual needs or empirical values. For example, the preset number could be twice the number of input channels before expansion.

[0147] In this system, the spatially aware feedforward network takes the feature map from the previous multi-head self-attention layer as input and outputs a feature map with the same shape as the previous feature map. The spatially aware feedforward network is an improved feedforward network structure that integrates multi-scale deep convolution and spatial gating mechanisms. It is primarily used to enhance the spatial awareness and local context modeling capabilities of the feedforward stage in the feature processing network.

[0148] In some embodiments of this application, the first multi-scale deep convolution module is specifically used to: divide the input ninth feature map into multiple eighteenth feature maps in the channel dimension, and then concatenate the multiple eighteenth feature maps after performing deep convolution with different kernel sizes to obtain a first concatenated feature map, and then process the first concatenated feature map with the ninth feature map through residual connection and ReLU function to output the eleventh feature map; The second multi-scale deep convolution module is specifically used to: divide the input fifteenth feature map into multiple nineteenth feature maps in the channel dimension, and then concatenate the multiple nineteenth feature maps after performing deep convolution with different kernel sizes to obtain a second concatenated feature map. The second concatenated feature map is then residually connected to the fifteenth feature map and processed by the ReLU function to output the sixteenth feature map.

[0149] The kernel sizes for depthwise convolutions of multiple 18th feature maps are different. Similarly, the kernel sizes for depthwise convolutions of multiple 19th feature maps are also different.

[0150] In this embodiment, the spatially aware feedforward network normalizes the feature map from the previous multi-head self-attention layer and restores it to a two-dimensional spatial structure. Then, it is divided into two parts along the channel dimension: one part serves as a spatial enhancement branch, extracting local texture and edge information through a multi-scale depthwise convolution (MSD) module. Using depthwise convolutions with different kernel sizes, it can fuse feature information from different receptive fields. The other part retains the original semantics to maintain global consistency in feature representation. The two feature parts are re-concatenated along the channel dimension and flattened back to a sequence form. Subsequently, a first linear transformation is applied to expand the hidden dimension (expanding the number of input channels to a preset number), and the signal is divided into a gated branch and a value branch. The gated branch, after restoring the spatial shape, is input to the multi-scale depthwise convolution module to fuse multi-scale feature information. Finally, it generates multi-scale response-enhanced features through residual connections and the ReLU function. The value branch maintains the linear mapping structure and carries basic channel semantic information. The two feature branches are multiplied element-wise to achieve spatial gating fusion, integrating spatial and channel information. Finally, a second linear transformation restores the original dimension, and a residual connection is made with the input features to form the final output. The entire process achieves end-to-end spatial feature extraction from global normalization, local spatial enhancement, multi-scale convolutional fusion to gating modulation, giving the feedforward network stage significant spatial modeling capabilities and structure awareness, providing the detection network with stronger target region response and context-dependent expression.

[0151] like Figure 6 The diagram shown is a schematic representation of the spatial sensing feedforward network provided in an embodiment of this application. Figure 6 The diagram shown is a structural schematic of the spatial channel distillation enhancement module provided in the embodiment of this application. Figure 6 In this diagram, LN represents layer normalization, MSDBlock represents a multi-scale deep convolutional module, Linear represents linear transformation, Reshape represents a reshaping unit, split represents a partitioning unit, DWConv represents deep convolution, k=3 represents a 3×3 convolutional kernel, k=5 represents a 5×5 convolutional kernel, k=7 represents a 7×7 convolutional kernel, k=9 represents a 9×9 convolutional kernel, and add represents a residual connection. Let the feature map of the preceding multi-head self-attention layer of the spatially aware feedforward network be denoted as... .in, Indicates the sequence length. , Indicates batch size, This represents the number of input channels for the corresponding feature map. and This represents the spatial resolution of the corresponding feature map. Based on this, combined with... Figure 6 The schematic diagram of the spatial sensing feedforward network shown illustrates the main data processing flow of the forward propagation process of the spatial sensing feedforward network, which can be represented as follows: The first processing module performs layer normalization on the input feature map to obtain the feature map. The calculation formula can be expressed as follows:

[0152] The feature map obtained after layer normalization is restored to a two-dimensional spatial structure. The calculation formula can be expressed as follows:

[0153] Feature map The ninth feature map partitioned along the channel dimension. and the tenth feature map The calculation formula can be expressed as follows:

[0154] in, .

[0155] The twelfth feature map is output after the ninth feature map is processed sequentially through the first multi-scale depthwise convolution module and the concatenation module. The calculation formula can be expressed as follows:

[0156] The feature map obtained by expanding the twelfth feature map back into sequence form The calculation formula can be expressed as follows:

[0157] The feature map obtained after channel expansion and activation of the twelfth feature map The calculation formula can be expressed as follows:

[0158] in, This represents the weight matrix of the first linear transformation.

[0159] In terms of channel dimension Divided into the thirteenth feature map and the fourteenth feature map Feature fusion is achieved through gating operations:

[0160] in, This represents the sixteenth feature map; This represents the seventeenth feature map.

[0161] After linear transformation and residual connection, the output feature map It can be represented as follows:

[0162] in, This represents the weight matrix of the second linear transformation.

[0163] Assume the feature map input to the multi-scale depthwise convolutional module (i.e., the first multi-scale depthwise convolutional module or the second multi-scale depthwise convolutional module) is... The output feature map is The processing procedure of the multi-scale depthwise convolution module can be represented as follows:

[0164] in, , , , These represent the four sub-feature maps that divide the input feature map along the channel dimension (i.e., when the input feature map is the ninth feature map, these four sub-feature maps are the eighteenth feature map; when the input feature map is the fifteenth feature map, these four sub-feature maps are the nineteenth feature map). The values ​​are 3, 5, 7, and 9; This represents the stitched feature map obtained by concatenating these four sub-feature maps after performing depthwise convolutions with four different kernel sizes.

[0165] This application proposes a method for detecting whether workers are wearing protective equipment (PE) in construction scenarios, based on a PE network. While maintaining end-to-end structural simplicity, this method systematically improves three key aspects: feature extraction, feature fusion, and sample matching. It can effectively identify various types of PE, including safety helmets, safety vests, gloves, and goggles, in complex construction environments, significantly improving the robustness and accuracy of the detection.

[0166] First, a spatial channel distillation enhancement module is proposed in the backbone network feature extraction stage. Through a dual distillation enhancement mechanism of spatial and channel features, efficient fusion and saliency guidance of multi-scale features are achieved. This module can adaptively enhance the response of salient regions related to the protected parts, suppress irrelevant interference in the construction background (such as machinery and scaffolding), and improve the discriminability of small targets (such as glasses and gloves) and the structural consistency of large targets (such as safety vests) without significantly increasing computational load. Second, a spatially perceptive feedforward network is adopted to replace the traditional feedforward network in the encoder-decoder feedforward stage. The spatially perceptive feedforward network achieves a dynamic balance between local texture and global semantics through multi-scale deep convolution and spatial gating mechanisms. Larger receptive field branches can capture information about the overall geometric structure of the human body, while smaller-scale branches retain detailed features of key parts of the body such as the head and hands, thus giving the personal protective equipment network a stronger ability to discriminate the wearing status of protective equipment. Finally, an improved matching cost and soft matching mechanism are proposed for the positive and negative sample matching strategy. Building upon traditional Hungarian matching, a comprehensive cost function based on center distance and shape constraints is introduced, enabling the model to pay greater attention to geometric consistency and scale matching between targets when assigning positive samples. Simultaneously, a multi-positive-sample soft allocation strategy is employed, assigning multiple predicted samples with different weights to each real target, mitigating the training instability that may arise from single matching. This effectively improves the model's robustness to matching under occlusion, blur, and complex lighting conditions, accelerating convergence and increasing recall.

[0167] In summary, this application's embodiments, through collaborative optimization at three levels—feature extraction, feature representation, and sample matching—enable the personal protective equipment (PPE) network to possess excellent detection performance for PPE of different scales and locations in complex construction scenarios. This method not only significantly improves the recognition accuracy of protective equipment such as safety helmets and vests but also enhances the model's generalization ability under conditions of multi-target coexistence, changing viewpoints, and high-resolution input, providing an efficient, reliable, and scalable intelligent detection solution for construction safety supervision.

[0168] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0169] Corresponding to the deep learning-based protective equipment wearing detection method described in the above embodiments, Figure 7 A schematic diagram of the structure of the protective equipment wearing detection device based on deep learning provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0170] Reference Figure 7 The device includes: Image acquisition module 701 is used to acquire an image to be detected, wherein the image to be detected includes at least one key part of the human body of the worker; The target detection module 702 is used to input the image to be detected into the personal protective equipment network to obtain a first detection result for the protective equipment and a second detection result for the key parts of the human body. The status determination module 703 is used to determine the protective equipment wearing status of the worker based on the first detection result and the second detection result; The personal protective equipment network includes a backbone network and a feature processing network, wherein the backbone network includes at least one layer of spatial channel distillation enhancement module; The spatial channel distillation enhancement module in each layer is used to: perform joint enhancement of the spatial dimension and channel dimension on the input first feature map, and output the second feature map; The feature processing network is used to: determine the first detection result and the second detection result based on the second feature map output by the last layer of the spatial channel distillation enhancement module.

[0171] In some embodiments, the spatial channel distillation enhancement module includes: a spatial attention submodule, multiple first convolutional layers, a distillation submodule, a fusion submodule, and a channel attention submodule; The spatial attention submodule is used to: enhance the spatial attention of the input first feature map and output a third feature map; The first convolutional layer is used to: perform multi-level local feature extraction, channel expansion and channel restoration on the input third feature map, and output a fourth feature map; The distillation submodule is used to: perform information distillation from the local space to the channel dimension on the input third feature map, and output a fifth feature map; The fusion submodule is used to: fuse the input fourth feature map and the fifth feature map, and output a fused feature map; The channel attention submodule is used to: perform channel attention enhancement on the input fused feature map and output the second feature map.

[0172] In some embodiments, the distillation submodule includes: a second convolutional layer, a local spatial distiller, and a third convolutional layer; The second convolutional layer is used to: perform a projection transformation on the input third feature map and output a sixth feature map; The local spatial distiller is used to: perform local spatial importance distillation on the input sixth feature map, map it to the channel dimension to restore it to a two-dimensional spatial structure, upsample it, and output a reconstructed feature map; The third convolutional layer is used to: adjust the channels of the input reconstructed feature map and output the fifth feature map.

[0173] In some embodiments, the local spatial distiller includes: a splitting unit, a reshaping unit, and a processing unit; The segmentation unit is used to: segment the input sixth feature map into multiple non-overlapping segments in the spatial dimension. Local blocks, Integers greater than zero; The reshaping unit is used to: rearrange the input multiple local blocks to obtain a rearranged feature map, wherein the dimension of the sixth feature map is... The dimension of the rearranged feature map is , Indicates batch size, Indicates the number of input channels. and Indicates spatial resolution; The processing unit is configured to: based on the rearranged feature map, calculate the mean of any rearranged local block in the channel dimension, process it through a feedforward network to obtain the corresponding seventh feature map, process the seventh feature map through a softmax function to obtain a local attention weight matrix, multiply the local attention weight matrix with learnable parameters to obtain an adjusted weight matrix, multiply the adjusted weight matrix element-wise with the seventh feature map to obtain an eighth feature map, restore the eighth feature map to a two-dimensional spatial structure, perform upsampling, and output the reconstructed feature map.

[0174] In some embodiments, the feature processing network includes a Transformer model and a detection head, wherein a spatially aware feedforward network is introduced into the Transformer model to replace the original feedforward network; The spatial perception feedforward network includes a first processing module, a first multi-scale deep convolution module, a stitching module, a second processing module, a second multi-scale deep convolution module, and a third processing module. The first processing module is used to: perform layer normalization on the input feature map and restore it to a two-dimensional spatial structure, and divide it into a ninth feature map and a tenth feature map in the channel dimension; The first multi-scale deep convolution module is used to: extract and fuse features at different scales from the input ninth feature map, and output the eleventh feature map; The splicing module is used to: splice the input tenth feature map and the eleventh feature map to output the twelfth feature map; The second processing module is used to: expand the input twelfth feature map back into sequence form, perform channel expansion, divide it into a thirteenth feature map and a fourteenth feature map in the channel dimension, restore the thirteenth feature map to a two-dimensional spatial structure, and output the fifteenth feature map; The second multi-scale deep convolution module is used to: extract and fuse features at different scales from the input fifteenth feature map, and output the sixteenth feature map; The third processing module is used to: expand the input sixteenth feature map back to sequence form and multiply it element-wise with the fourteenth feature map to obtain the seventeenth feature map; restore the seventeenth feature map to its original dimension and perform a residual connection with the feature map input to the first processing module, wherein the original dimension is the dimension of the feature map input to the first processing module.

[0175] In some embodiments, the first multi-scale deep convolution module is specifically used to: divide the input ninth feature map into multiple eighteenth feature maps in the channel dimension, and concatenate the multiple eighteenth feature maps after performing deep convolution with different kernel sizes to obtain a first concatenated feature map, and then process the first concatenated feature map and the ninth feature map with a residual connection and then process it with the ReLU function to output the eleventh feature map; The second multi-scale deep convolution module is specifically used to: divide the input fifteenth feature map into multiple nineteenth feature maps in the channel dimension, and then concatenate the multiple nineteenth feature maps after performing deep convolution with different kernel sizes to obtain a second concatenated feature map. The second concatenated feature map is then residually connected to the fifteenth feature map and processed by the ReLU function to output the sixteenth feature map.

[0176] In some embodiments, the above-described apparatus further includes: The first acquisition module is used to acquire a predicted target set and a real target set of the sample image; the predicted target set includes at least one predicted bounding box of the key human body part, and the real target set includes at least one real bounding box of the key human body part; The second acquisition module is used to acquire, for any predicted bounding box and any ground truth bounding box, the generalized intersection-over-union ratio between the ground truth bounding box and the predicted bounding box, the probability that the predicted bounding box belongs to the category to which the ground truth bounding box belongs, the length of the diagonal of the envelope of the ground truth bounding box and the predicted bounding box, and the width and height of the envelope. The first calculation module is used to calculate the center distance constraint between the real box and the predicted box based on the length of the diagonal, the coordinates of the center point of the predicted box and the coordinates of the center point of the real box. The center distance constraint is used to constrain the proximity between the center point of the real box and the center point of the predicted box. The second calculation module is used to calculate the shape constraint between the real box and the predicted box based on the width and height of the predicted box, the width and height of the ground truth box, and the width and height of the envelope box. The shape constraint is used to constrain the similarity in shape between the real box and the predicted box. The third calculation module is used to calculate the matching cost between the real box and the predicted box based on the probability that the predicted box belongs to the category of the real box, the generalized intersection-union ratio, the center distance constraint, and the shape constraint. The network training module is used to calculate the loss value of the sample image based on the probability that the predicted box belongs to the category of the real box, and all matching costs between the predicted target set and the real target set, and to train the personal protective equipment network based on the loss value.

[0177] In some embodiments, the loss value of the sample image includes the regression loss value and the classification loss value of the sample image; the network training module described above is specifically used for: For any real bounding box in the set of real targets, determine K candidate bounding boxes that match the real bounding box from the set of predicted targets; the K candidate bounding boxes that match the real bounding box refer to the K predicted bounding boxes with the minimum matching cost with the real bounding box; K is an integer greater than 1; Based on the matching cost between the ground truth bounding box and the K candidate bounding boxes, calculate the weight between the ground truth bounding box and the K candidate bounding boxes; The regression loss value of the sample image is calculated based on the regression loss value and weight between each ground truth bounding box and the K candidate bounding boxes. Based on the weights between each ground truth bounding box and each predicted bounding box, and the probability that each predicted bounding box belongs to the category of each ground truth bounding box, the classification loss value of the sample image is calculated. For any ground truth bounding box and the remaining bounding box, the weight between the ground truth bounding box and the remaining bounding box is a preset value. The remaining bounding box is the predicted bounding box that does not belong to the K candidate bounding boxes that match the ground truth bounding box.

[0178] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0179] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 8 of this embodiment includes: at least one processor 80 ( Figure 8(Only one is shown in the diagram), memory 81, and computer program 82 stored in said memory 81 and executable on said at least one processor 80, wherein said processor 80 executes said computer program 82 to implement the steps in any of the above method embodiments.

[0180] The electronic device may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of electronic device 8 and does not constitute a limitation on electronic device 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0181] The processor 80 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0182] In some embodiments, the memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. In other embodiments, the memory 81 may be an external storage device of the electronic device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 8. Furthermore, the memory 81 may include both internal and external storage units of the electronic device 8. The memory 81 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0183] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0184] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0185] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0186] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0187] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0188] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0189] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting the wearing of protective equipment based on deep learning, characterized in that, include: Acquire an image to be detected, wherein the image to be detected includes at least one key part of the worker's body; The image to be detected is input into the personal protective equipment network to obtain a first detection result for the protective equipment and a second detection result for the key parts of the human body. Based on the first detection result and the second detection result, the protective equipment wearing status of the worker is determined; The personal protective equipment network includes a backbone network and a feature processing network, wherein the backbone network includes at least one layer of spatial channel distillation enhancement module; The spatial channel distillation enhancement module in each layer is used to: perform joint enhancement of the spatial dimension and channel dimension on the input first feature map, and output the second feature map; The feature processing network is used to: determine the first detection result and the second detection result based on the second feature map output by the last layer of the spatial channel distillation enhancement module.

2. The protective equipment wearing detection method according to claim 1, characterized in that, The spatial channel distillation enhancement module includes: a spatial attention submodule, a multi-layer first convolutional layer, a distillation submodule, a fusion submodule, and a channel attention submodule; The spatial attention submodule is used to: enhance the spatial attention of the input first feature map and output a third feature map; The first convolutional layer is used to: perform multi-level local feature extraction, channel expansion and channel restoration on the input third feature map, and output a fourth feature map; The distillation submodule is used to: perform information distillation from the local space to the channel dimension on the input third feature map, and output a fifth feature map; The fusion submodule is used to: fuse the input fourth feature map and the fifth feature map, and output a fused feature map; The channel attention submodule is used to: perform channel attention enhancement on the input fused feature map and output the second feature map.

3. The protective equipment wearing detection method according to claim 2, characterized in that, The distillation submodule includes: a second convolutional layer, a local spatial distiller, and a third convolutional layer; The second convolutional layer is used to: perform a projection transformation on the input third feature map and output a sixth feature map; The local spatial distiller is used to: perform local spatial importance distillation on the input sixth feature map, map it to the channel dimension to restore it to a two-dimensional spatial structure, perform upsampling, and output a reconstructed feature map; The third convolutional layer is used to: adjust the channels of the input reconstructed feature map and output the fifth feature map.

4. The protective equipment wearing detection method according to claim 3, characterized in that, The local space distiller includes: a cutting unit, a reshaping unit, and a processing unit; The segmentation unit is used to: segment the input sixth feature map into multiple non-overlapping segments in the spatial dimension. Local blocks, Integers greater than zero; The reshaping unit is used to: rearrange the input multiple local blocks to obtain a rearranged feature map, wherein the dimension of the sixth feature map is... The dimension of the rearranged feature map is , Indicates batch size, Indicates the number of input channels. and Indicates spatial resolution; The processing unit is configured to: based on the rearranged feature map, calculate the mean of any rearranged local block in the channel dimension, process it through a feedforward network to obtain the corresponding seventh feature map, process the seventh feature map through a softmax function to obtain a local attention weight matrix, multiply the local attention weight matrix with learnable parameters to obtain an adjusted weight matrix, multiply the adjusted weight matrix element-wise with the seventh feature map to obtain an eighth feature map, restore the eighth feature map to a two-dimensional spatial structure, perform upsampling, and output the reconstructed feature map.

5. The method for detecting the wearing of protective equipment according to claim 1, characterized in that, The feature processing network includes a Transformer model and a detection head. A spatially aware feedforward network is introduced into the Transformer model to replace the original feedforward network. The spatial perception feedforward network includes a first processing module, a first multi-scale deep convolution module, a stitching module, a second processing module, a second multi-scale deep convolution module, and a third processing module. The first processing module is used to: perform layer normalization on the input feature map and restore it to a two-dimensional spatial structure, and divide it into a ninth feature map and a tenth feature map in the channel dimension; The first multi-scale deep convolution module is used to: extract and fuse features at different scales from the input ninth feature map, and output the eleventh feature map; The splicing module is used to: splice the input tenth feature map and the eleventh feature map to output the twelfth feature map; The second processing module is used to: expand the input twelfth feature map back into sequence form, perform channel expansion, divide it into a thirteenth feature map and a fourteenth feature map in the channel dimension, restore the thirteenth feature map to a two-dimensional spatial structure, and output the fifteenth feature map; The second multi-scale deep convolution module is used to: extract and fuse features at different scales from the input fifteenth feature map, and output the sixteenth feature map; The third processing module is used to: expand the input sixteenth feature map back to sequence form and multiply it element-wise with the fourteenth feature map to obtain the seventeenth feature map; restore the seventeenth feature map to its original dimension and perform a residual connection with the feature map input to the first processing module, wherein the original dimension is the dimension of the feature map input to the first processing module.

6. The method for detecting the wearing of protective equipment according to claim 5, characterized in that, The first multi-scale deep convolution module is specifically used to: divide the input ninth feature map into multiple eighteenth feature maps in the channel dimension, and perform deep convolution with different kernel sizes on the multiple eighteenth feature maps respectively, and then concatenate them to obtain a first concatenated feature map; after performing residual connection between the first concatenated feature map and the ninth feature map, the eleventh feature map is output after processing by the ReLU function. The second multi-scale deep convolution module is specifically used to: divide the input fifteenth feature map into multiple nineteenth feature maps in the channel dimension, and then concatenate the multiple nineteenth feature maps after performing deep convolution with different kernel sizes to obtain a second concatenated feature map. The second concatenated feature map is then residually connected to the fifteenth feature map and processed by the ReLU function to output the sixteenth feature map.

7. The method for detecting the wearing of protective equipment according to any one of claims 1 to 6, characterized in that, Before inputting the image to be detected into the personal protective equipment network, the method further includes: Obtain a set of predicted targets and a set of real targets for a sample image; the set of predicted targets includes at least one predicted bounding box for a key part of the human body, and the set of real targets includes at least one real bounding box for a key part of the human body. For any predicted bounding box and any ground truth bounding box, obtain the generalized intersection-over-union ratio between the ground truth bounding box and the predicted bounding box, the probability that the predicted bounding box belongs to the category to which the ground truth bounding box belongs, the length of the diagonal of the envelope of the ground truth bounding box and the predicted bounding box, and the width and height of the envelope. Based on the length of the diagonal, the coordinates of the center point of the predicted bounding box, and the coordinates of the center point of the ground truth bounding box, a center distance constraint between the ground truth bounding box and the predicted bounding box is calculated. The center distance constraint is used to constrain the proximity between the center point of the ground truth bounding box and the center point of the predicted bounding box. Based on the width and height of the predicted bounding box, the width and height of the ground truth bounding box, and the width and height of the envelope bounding box, the shape constraint between the ground truth bounding box and the predicted bounding box is calculated. The shape constraint is used to constrain the similarity between the ground truth bounding box and the predicted bounding box in shape. Based on the probability that the predicted bounding box belongs to the category of the ground truth bounding box, the generalized intersection-union ratio, the center distance constraint, and the shape constraint, the matching cost between the ground truth bounding box and the predicted bounding box is calculated; Based on the probability that the predicted bounding box belongs to the category of the ground truth bounding box, and all matching costs between the predicted target set and the ground truth target set, the loss value of the sample image is calculated, and the personal protective equipment network is trained based on the loss value.

8. The method for detecting the wearing of protective equipment according to claim 7, characterized in that, The loss value of the sample image includes the regression loss value and the classification loss value of the sample image; the calculation of the loss value of the sample image based on the probability that the predicted box belongs to the category of the ground truth box, and all matching costs between the predicted target set and the ground truth target set, includes: For any real bounding box in the set of real targets, determine K candidate bounding boxes that match the real bounding box from the set of predicted targets; the K candidate bounding boxes that match the real bounding box refer to the K predicted bounding boxes with the minimum matching cost with the real bounding box; K is an integer greater than 1; Based on the matching cost between the ground truth bounding box and the K candidate bounding boxes, calculate the weight between the ground truth bounding box and the K candidate bounding boxes; The regression loss value of the sample image is calculated based on the regression loss value and weight between each ground truth bounding box and the K matched candidate bounding boxes. Based on the weights between each ground truth bounding box and each predicted bounding box, and the probability that each predicted bounding box belongs to the category of each ground truth bounding box, the classification loss value of the sample image is calculated. For any ground truth bounding box and the remaining bounding box, the weight between the ground truth bounding box and the remaining bounding box is a preset value. The remaining bounding box is the predicted bounding box that does not belong to the K candidate bounding boxes that match the ground truth bounding box.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the protective equipment wearing detection method as described in any one of claims 1 to 8.

10. A computer program product, characterized in that, Includes a computer program, which, when run, causes the protective equipment wearing detection method as described in any one of claims 1 to 8 to be performed.

Images