A safety helmet wearing detection method in a complex scene

Abstract

The application discloses a safety helmet wearing detection method based on multi-scale features, which comprises the following steps: introducing an attention mechanism in the YOLO v5 network backbone part to reduce the loss of effective information in the network during transmission; adding a fourth detection scale 104x104 in the neck and head of the YOLO v5 network to enhance the detection capability for small targets; after pre-training the CSPDarkNet53 model on a large data set, migrating the feature extraction capability of the CSPDarkNet53 model to the safety helmet wearing detection model to alleviate the problem of insufficient data set; inferring the human body bounding box according to the safety helmet wearing detection box, extracting the skeleton key points of the personnel without wearing safety helmets, designing a gait recognition module, and identifying the identity of the personnel without wearing safety helmets; the application improves the accuracy of the safety helmet wearing detection model in a complex scene by using multi-scale features, and realizes the confirmation of the identity of the personnel without wearing safety helmets by fusing the gait recognition algorithm.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and target detection technology, and in particular to a method for detecting helmet wearing based on multi-scale features and its application. Background Technology

[0002] With urbanization, the construction of massive infrastructure projects such as subways has brought increasing attention to safety issues at construction sites. Safety helmets are one of the most effective personal protective equipment for reducing worker injuries from falls or impacts from falling objects, and wearing them on construction sites is a global legal requirement. However, due to the discomfort of wearing safety helmets and a lack of safety awareness among workers, they often unconsciously remove them. Therefore, long-term monitoring of whether workers are wearing safety helmets correctly is crucial for safe production and can improve safety management. Traditional safety helmet checks at construction sites mainly include monitoring image checks and manual patrol checks. The latter requires a significant amount of time and labor, while manual video monitoring requires inspectors to stare at screens for extended periods, which can lead to misjudgments due to fatigue. Against this backdrop, new technologies that utilize sensors and image analysis to detect the helmet wearing status of construction workers have rapidly developed.

[0003] Various sensors are used for helmet wearing detection, such as chin strap sensors, triaxial accelerometer sensors, RFID, and pressure sensors. However, these methods not only increase the investment in detection, but workers also perceive these sensors as an intrusion. Therefore, due to privacy and health concerns, workers are often unwilling to wear helmets equipped with these sensors. Thus, non-invasive methods are more readily accepted, and a common approach is to use image sensors to detect helmet wearing at construction sites. Many image analysis algorithms have been proposed for target detection, such as Gaussian mixture models, histograms of oriented gradients (HOR), and support vector machines (SVMs). These algorithms have also been used for helmet detection, such as using skin color and Hough circles to identify helmets, using HOR to detect workers and then using color and Hough circles to complete helmet detection, or using SVMs to classify worker helmet wearing status. However, all of these traditional detection methods have their drawbacks. Gaussian mixture models cannot accurately distinguish foreground categories, and manually designed features such as color and HOR are difficult to effectively capture high- and mid-level features. Therefore, detectors based on these methods have poor generalization ability and are difficult to apply in practical engineering.

[0004] With the development of computer technology, the application of GPUs in large-scale parallel computing has made it possible to train large, deep neural networks. In the field of object detection, a series of deep learning-based methods have been developed, among which convolutional neural networks (CNNs) have become the most commonly used method due to their superiority in high-level feature extraction. Therefore, they have gradually replaced traditional detection methods in image analysis. Generally speaking, object detection methods based on convolutional neural networks can be divided into two categories: one is region-based two-step detectors, such as R-CNN, Fast R-CNN, and Faster R-CNN; the other is regression-based single-step detectors, such as YOLO, SSD, and RetinalNet. Generally, compared with two-step detectors, single-step detectors have higher detection speed but lower accuracy. How to balance speed and accuracy is a problem that needs to be considered when using deep learning object detection algorithms to detect helmet wearing.

[0005] Meanwhile, conducting helmet detection at subway construction sites presents challenges such as scale variations, perspective distortion, and small target factors. For example, varying distances from the camera can cause differences in the size of individuals being detected in the image; densely packed construction workers in some scenarios may obscure each other's view; due to limitations, cameras need to be placed far from the construction site, resulting in smaller-sized individuals in the image; and complex backgrounds at some construction sites can also affect helmet detection. These challenges significantly limit the performance of detection algorithms.

[0006] This invention aims to expand the detectable range of a target detection algorithm by employing four detection scales to address the size variations of targets in an image. Simultaneously, it generates more reasonable prediction boxes within a single image region to encompass as many targets as possible, mitigating missed detections due to occlusion. For small targets, an attention mechanism is introduced to focus the algorithm on key areas of the image, while the fourth detection scale is designed to have a larger receptive field specifically for small target detection. Finally, data augmentation and transfer learning are used to improve the network's generalization ability to handle complex backgrounds. After achieving good detection results, this invention plans to utilize gait recognition to identify individuals not wearing helmets. Summary of the Invention

[0007] This invention addresses the issue of single-step target detectors being fast but inaccurate by proposing a method that utilizes multi-scale features for detection. This method enables accurate and rapid detection of helmet wearing status and identification of individuals not wearing helmets. The specific steps of this invention are as follows:

[0008] Step A: Based on the characteristic that the size of the target changes greatly in the image under complex scenes, four feature scales are designed for safety helmet detection;

[0009] Based on the YOLO v5 network's detection of the neck and head, a fourth detection scale with a smaller receptive field is added to enhance the detection effect of small targets. The four additional detection scales are 13×13, 26×26, 52×52, and 104×104, which have a larger scale detection range compared to the original three scales.

[0010] During YOLO v5 training, the objective function of its bounding box regression is related to the true value B. gt It is related to the predicted value B, and its calculation is shown in formula (1).

[0011]

[0012] Where d is the center of the true value c gt The distance between the center c of the predicted box and the ground truth is l, which is the diagonal length of the minimum bounding rectangle of the predicted box and the ground truth. IoU is the intersection-union ratio of the two rectangles, and its calculation is shown in formula (2).

[0013]

[0014] α is a weighting parameter, defined as shown in formula (3).

[0015]

[0016] v represents aspect ratio consistency, based on the true value and the width w of the predicted bounding box. gt w and high h gt h is calculated as shown in formula (4).

[0017]

[0018] Based on the loss function calculation formula, it can be determined that generating more suitable prediction boxes can improve the training effect of the network.

[0019] Step B: Introduce an attention mechanism into the YOLO v5 backbone network and design an improved YOLO v5 network to reduce the loss when extracting effective information from lower layers to higher layers.

[0020] Introducing a fourth detection scale leads to a deeper network layer, and information loss occurs when information from lower layers is transmitted to higher layers. Therefore, an attention mechanism is introduced to reduce information loss, improve transmission efficiency, and enable the network to notice small targets in the image.

[0021] For convolutional mapping F tr :X→U,X∈R H′×W′×C′ ,U∈R H×W×C v c If the c-th convolutional kernel is represented, then the output U = [u1, u2, ..., u...]. C As shown in equation (5),

[0022] u c =v c *X (5)

[0023] The introduced attention mechanism includes squeezing and excitation operations. The squeezing operation encodes the spatial features of a channel into a global feature, which is implemented using global average pooling, as shown in Equation (6).

[0024]

[0025] The activation operation uses the sigmoid(σ) activation function as shown in equation (7).

[0026] s = F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z)) (7)

[0027] Where δ is the ReLU activation function. r is the dimensionality reduction ratio; finally, the learned channel weights are assigned to the original features to obtain the final output as shown in formula (8).

[0028] x = s·u (8)

[0029] The attention mechanism makes the model pay more attention to the channel features with the most information and suppress unimportant channel features. This invention introduces two SE modules, which are introduced before the second and third convolutional concatenation operations in the backbone network.

[0030] Step C involves designing evaluation indicators from both accuracy and speed perspectives to assess the detection performance, and performing targeted data augmentation and CutMix data enhancement for categories with poor detection performance.

[0031] To compare the improvements brought about by the model improvement, the improved model and the original model were trained and tested on the same training and test sets, and evaluated from two aspects: accuracy and speed. In terms of accuracy, the metrics precision and recall were calculated and the PR curve was plotted. The mean precision (MAP) was calculated, and the calculation formulas are shown in formulas (9), (10) and (11).

[0032]

[0033]

[0034]

[0035] Where True Positive (TP) is the number of targets correctly predicted by the model, False Positive (FN) is the number of other objects incorrectly predicted as targets, and False Negative (FN) is the number of true values ​​that were not correctly predicted by the model; n is the number of classes, and for each class, the Average Precision (AP) is... i The area under the corresponding PR curve is denoted as PR; in terms of speed, the average detection time on the same test set is used as the indicator; after the evaluation, targeted data augmentation and CutMix data enhancement are performed on categories with poor detection performance.

[0036] Let x∈R W×H×C Let y and y represent the training objective and its label, respectively. Then, the CutMix data augmentation operation for images A and B is shown in formulas (12) and (13).

[0037]

[0038]

[0039] Where M∈{0,1} W×H is a mask matrix marked with 0s and 1s, which is the same size as the image, where 0 represents the region to be cropped and 1 represents the region to be retained; ⊙ represents element-wise multiplication, and λ is a uniform distribution following (0,1);

[0040] Data augmentation can enable models to focus on features that are difficult to distinguish from the target, thereby improving training efficiency;

[0041] Step D: Based on the improved YOLO v5, transfer learning is performed on the feature extraction capabilities of its backbone network CSPDarkNet53 model to achieve helmet recognition in complex scenarios.

[0042] After the improved model achieves good results on the helmet detection dataset, transfer learning of models on other similar detection tasks can further enhance the generalization ability of the helmet detection model.

[0043] The general definition of transfer learning is as follows:

[0044] Condition: Given a source domain D s Learning task T on the source domain s Target domain D t and learning task T in the target domain t

[0045] Objective: To utilize D s and T s Learn the prediction function f(·) over the target domain.

[0046] Constraints: D s ≠D t Ts ≠T t

[0047] This invention utilizes the transfer learning of the backbone network CSPDarkNet53, which was initially trained on large datasets such as MS COCO and completed the corresponding detection tasks. All its convolutional layers and fully connected layers before classification are frozen and then ported to the improved model. The model is then retrained on the dataset enhanced in step C to obtain a helmet wearing detection model that utilizes multi-scale features. This model detects the helmet wearing status of workers at construction sites and outputs the detection results, which are divided into two categories: not wearing a helmet (person) and wearing a helmet (hat).

[0048] Step E: Based on the detection results of the improved YOLO v5, design a gait recognition algorithm module to identify individuals not wearing helmets.

[0049] To reduce computational load, gait recognition is performed only on the person detection results. For the top left corner coordinates (x1, y1) and bottom right corner coordinates (x2, y2) of the detection box, the bounding box of the person not wearing a helmet can be inferred by calculating the width and height of the head, as shown in formulas (14)-(19).

[0050] h = y2 - y1 (14)

[0051] w = x2 - x1 (15)

[0052] x3 = max(0, x1 - 2 × w) (16)

[0053] y3=min(0,y1-h) (17)

[0054] x4 = min(width, x2 + 2 × w) (18)

[0055] y4=min(height,y2+5×h) (19)

[0056] Where width and height are the width and height of the image, (x3, y3) and (x4, y4) are the coordinates of the top left and bottom right corners of the human bounding box; 17 skeleton key points are extracted from the bounding box using HRNet, and the gait sequence of the skeleton key points is input into the gait recognition network GaitGraph for recognition.

[0057] The human skeleton can be structurally represented as G=(V,ε), where V={v1,…,v n Let} represent the set of N nodes of a joint, and ε represent the set of edges of a bone as represented by an adjacency matrix, which can be represented as A∈R. N×N When vi to v j When there are edges connecting them, A i,j =1, otherwise A i,j =0, because G is non-directional, so A is a symmetric matrix;

[0058] Human gait can be characterized as X = {x} t,n ∈R c |t,n∈Z,1≤t≤T,1≤n≤N}, for a feature tensor X∈R T×N×C , where x t,n =X t,n For node v n In the C-dimensional feature vector at time t, there are a total of T frames, X t ∈R N×C At time t, the pose feature X in dimension C represents the corresponding 2D coordinates and its confidence level, and N is the number of nodes;

[0059] Therefore, the learnable weight matrix of network layer l can be expressed as: Its graph convolutional neural network layer update calculation for features at time t is as follows:

[0060]

[0061] in To add an identity matrix to the skeleton graph to ensure the characteristics of each node, for The diagonal matrix, where σ(·) is the activation function;

[0062] The gait recognition network of this invention consists of a ResGCN module, which is composed of a graph convolution, a regular temporal two-dimensional convolution, and a residual connection with a bottleneck structure. The entire network is composed of multiple ResGCN modules arranged in sequence. The output layer consists of an average pooling layer and a fully connected layer that generates feature vectors. Supervised contrastive loss is used as the loss function. The final output of the network is the encoding of the target and its corresponding feature vector.

[0063] The worker safety helmet wearing detection method based on multi-scale features of the present invention has the following advantages compared with the prior art:

[0064] 1. By extracting features from different layers of the network and splicing and fusing them, the target can be detected at multiple scales, thus achieving good detection performance for targets at multiple scales when detecting helmet wearing. This overcomes the shortcomings of previous helmet wearing detection algorithms that could not simultaneously handle multi-scale target detection.

[0065] 2. An attention mechanism was introduced before feature splicing in the network to reduce information loss caused by increasing the number of network layers, and to integrate more effective low-level feature information with high-level splicing to improve the helmet wearing detection effect.

[0066] 3. When performing targeted data augmentation, it is no longer necessary for the number of samples in each category to be roughly the same. Experimental results demonstrate that the model's detection performance across all categories has improved. The successful transfer learning of the CSPDarkNet53 model proves the effectiveness of the pre-trained model on the MS COCO dataset for the helmet-wearing detection task. Compared with other detection methods, this alleviates the problem of insufficient helmet datasets and, to some extent, reduces the workload of dataset annotation.

[0067] 4. Based on the detection results of helmet wearing status, a gait recognition module for people not wearing helmets was designed. After detecting the presence of people not wearing helmets, the module can further determine their identity and accurately remind them, thus further improving the helmet wearing detection framework based on deep learning. Attached Figure Description

[0068] Figure 1 This is a schematic diagram illustrating the loss function calculation during YOLO v5 training in this invention;

[0069] Figure 2 The diagram shows the improved YOLO v5 network model of this invention, where the improved parts are marked with red dashed lines. The markers at Backbone indicate the introduction of the attention mechanism, and the markers at Neck and Prediction indicate the addition of a fourth detection scale.

[0070] Figure 3 This is a flowchart of the helmet wearing detection based on multi-scale features in this invention. Detailed Implementation

[0071] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0072] Example 1

[0073] A method for detecting helmet wearing in complex scenarios includes the following steps:

[0074] Step A: Based on the characteristic that the size of the target changes greatly in the image under complex scenes, four feature scales are designed for safety helmet detection;

[0075] Based on the YOLO v5 network's detection of the neck and head, a fourth detection scale with a smaller receptive field is added to enhance the detection effect of small targets. The four additional detection scales are 13×13, 26×26, 52×52, and 104×104, which have a larger scale detection range compared to the original three scales.

[0076] During YOLO v5 training, the objective function of its bounding box regression is related to the true value B. gt It is related to the predicted value B, and its calculation is shown in formula (1).

[0077]

[0078] Where d is the center of the true value c gt The distance between the center c of the predicted box and the ground truth is l, which is the diagonal length of the minimum bounding rectangle of the predicted box and the ground truth. IoU is the intersection-union ratio of the two rectangles, and its calculation is shown in formula (2).

[0079]

[0080] α is a weighting parameter, defined as shown in formula (3).

[0081]

[0082] v represents aspect ratio consistency, based on the true value and the width w of the predicted bounding box. gt w and high h gt h is calculated as shown in formula (4).

[0083]

[0084] Based on the loss function calculation formula, it can be determined that generating more suitable prediction boxes can improve the training effect of the network.

[0085] Step B: Introduce an attention mechanism into the YOLO v5 backbone network and design an improved YOLO v5 network to reduce the loss when extracting effective information from lower layers to higher layers.

[0086] Introducing a fourth detection scale leads to a deeper network layer, and information loss occurs when information from lower layers is transmitted to higher layers. Therefore, an attention mechanism is introduced to reduce information loss, improve transmission efficiency, and enable the network to notice small targets in the image.

[0087] For convolutional mapping F tr :X→U,X∈R H′×w′×C′ ,U∈R H×W×c v c If the c-th convolutional kernel is represented, then the output U = [u1, u2, ..., u...]. C As shown in equation (5),

[0088] u c =v c *X (5)

[0089] The introduced attention mechanism includes squeezing and excitation operations. The squeezing operation encodes the spatial features of a channel into a global feature, which is implemented using global average pooling, as shown in Equation (6).

[0090]

[0091] The activation operation uses the sigmoid(σ) activation function as shown in equation (7).

[0092] s = F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z)) (7)

[0093] Where δ is the ReLU activation function. r is the dimensionality reduction ratio; finally, the learned channel weights are assigned to the original features to obtain the final output as shown in formula (8).

[0094] x = s·u (8)

[0095] The attention mechanism makes the model pay more attention to the channel features with the most information and suppress unimportant channel features. This invention introduces two SE modules, which are introduced before the second and third convolutional concatenation operations in the backbone network.

[0096] Step C involves designing evaluation indicators from both accuracy and speed perspectives to assess the detection performance, and performing targeted data augmentation and CutMix data enhancement for categories with poor detection performance.

[0097] To compare the improvements brought about by the model improvement, the improved model and the original model were trained and tested on the same training and test sets, and evaluated from two aspects: accuracy and speed. In terms of accuracy, the metrics precision and recall were calculated and the PR curve was plotted. The mean precision (MAP) was calculated, and the calculation formulas are shown in formulas (9), (10) and (11).

[0098]

[0099]

[0100]

[0101] Where True Positive (TP) is the number of targets correctly predicted by the model, False Positive (FN) is the number of other objects incorrectly predicted as targets, and False Negative (FN) is the number of true values ​​that were not correctly predicted by the model; n is the number of classes, and for each class, the Average Precision (AP) is... i The area under the corresponding PR curve is denoted as PR; in terms of speed, the average detection time on the same test set is used as the indicator; after the evaluation, targeted data augmentation and CutMix data enhancement are performed on categories with poor detection performance.

[0102] Let x∈R W×H×c Let y and y represent the training objective and its label, respectively. Then, the CutMix data augmentation operation for images A and B is shown in formulas (12) and (13).

[0103]

[0104]

[0105] Where M∈{0,1} W×H is a mask matrix marked with 0s and 1s, which is the same size as the image, where 0 represents the region to be cropped and 1 represents the region to be retained; ⊙ represents element-wise multiplication, and λ is a uniform distribution following (0,1);

[0106] Data augmentation can enable models to focus on features that are difficult to distinguish from the target, thereby improving training efficiency;

[0107] Step D: Based on the improved YOLO v5, transfer learning is performed on the feature extraction capabilities of its backbone network CSPDarkNet53 model to achieve helmet recognition in complex scenarios.

[0108] After the improved model achieves good results on the helmet detection dataset, transfer learning of models on other similar detection tasks can further enhance the generalization ability of the helmet detection model.

[0109] The general definition of transfer learning is as follows:

[0110] Condition: Given a source domain D s Learning task T on the source domain s Target domain D t and learning task T in the target domain t

[0111] Objective: To utilize D s and T s Learn the prediction function f(·) over the target domain.

[0112] Constraints: D s ≠D t Ts ≠T t

[0113] This invention utilizes the transfer learning of the backbone network CSPDarkNet53, which was initially trained on large datasets such as MS COCO and completed the corresponding detection tasks. All its convolutional layers and fully connected layers before classification are frozen and then ported to the improved model. The model is then retrained on the dataset enhanced in step C to obtain a helmet wearing detection model that utilizes multi-scale features. This model detects the helmet wearing status of workers at construction sites and outputs the detection results, which are divided into two categories: not wearing a helmet (person) and wearing a helmet (hat).

[0114] Step E: Based on the detection results of the improved YOLO v5, design a gait recognition algorithm module to identify individuals not wearing helmets.

[0115] To reduce computational load, gait recognition is performed only on the person detection results. For the top left corner coordinates (x1, y1) and bottom right corner coordinates (x2, y2) of the detection box, the bounding box of the person not wearing a helmet can be inferred by calculating the width and height of the head, as shown in formulas (14)-(19).

[0116] h = y2 - y1 (14)

[0117] w = x2 - x1 (15)

[0118] x3 = max(0, x1 - 2 × w) (16)

[0119] y3=min(0,y1-h) (17)

[0120] x4 = min(width, x2 + 2 × w) (18)

[0121] y4=min(height,y2+5×h) (19)

[0122] Where width and height are the width and height of the image, (x3, y3) and (x4, y4) are the coordinates of the top left and bottom right corners of the human bounding box; 17 skeleton key points are extracted from the bounding box using HRNet, and the gait sequence of the skeleton key points is input into the gait recognition network GaitGraph for recognition.

[0123] The human skeleton can be structurally represented as G=(V,ε), where V={v1,…,v n Let} represent the set of N nodes of a joint, and ε represent the set of edges of a bone as represented by an adjacency matrix, which can be represented as A∈R. N×N When vi to v j When there are edges connecting them, A i,j =1, otherwise A i,j =0, because G is non-directional, so A is a symmetric matrix;

[0124] Human gait can be characterized as X = {x} t,n ∈R c |t,n∈Z,1≤t≤T,1≤n≤N}, for a feature tensor X∈R T×N×C , where x t,n =X t,n For node v n In the C-dimensional feature vector at time t, there are a total of T frames, X t ∈R N×C At time t, the pose feature X in dimension C represents the corresponding 2D coordinates and its confidence level, and N is the number of nodes;

[0125] Therefore, the learnable weight matrix of network layer l can be expressed as: Its graph convolutional neural network layer update calculation for features at time t is as follows:

[0126]

[0127] in To add an identity matrix to the skeleton graph to ensure the characteristics of each node, for The diagonal matrix, where σ(·) is the activation function;

[0128] The gait recognition network of this invention consists of a ResGCN module, which is composed of a graph convolution, a regular temporal two-dimensional convolution, and a residual connection with a bottleneck structure. The entire network is composed of multiple ResGCN modules arranged in sequence. The output layer consists of an average pooling layer and a fully connected layer that generates feature vectors. Supervised contrastive loss is used as the loss function. The final output of the network is the encoding of the target and its corresponding feature vector.

[0129] Example 2

[0130] This embodiment uses the open-source safety helmet dataset Safety-Helmet-Wearing-Dataset and augmented images. Before targeted data augmentation (DA), it includes 9047 positive "hat" images (wearing helmets) and 9082 negative "person" images (not wearing helmets). The two categories are randomly divided into training and testing sets in an 8:2 ratio for training and testing the network. After targeted data augmentation, the number of "person" images increases to 35531, while the test set remains unchanged. To verify the effectiveness of the proposed modifications, in this embodiment, the original YOLO v5 network is selected as the baseline. A fourth detection scale (FS) is added, an attention mechanism (SB) is introduced, targeted data augmentation (DA) is performed, and transfer learning (PT) is applied. The model is tested on the same test set, and its accuracy and speed are evaluated. The experimental environment is shown in Table 1, and the experimental results are shown in Table 2.

[0131] Table 1 Experimental Environment

[0132]

[0133] Table 2 Experimental Results

[0134]

[0135] Comparative analysis shows that, compared to the baseline, each step of the network improvement method proposed in this invention can bring a significant improvement in accuracy. Ultimately, the accuracy reached 92.6% for APs wearing safety helmets and 91.7% for APs not wearing safety helmets, with an overall MAP of 92.2%. On the test set, it only takes an average of 3ms to complete the detection of an image, meeting the real-time requirements.

[0136] For the gait recognition module, a self-made dataset was used for testing. The dataset included 123 pre-modeled targets from 10 construction scenarios, totaling 1230 videos. Each video included both people wearing safety helmets and those not wearing them. The test required the algorithm to identify and recognize the individuals not wearing safety helmets. With the experimental platform unchanged, the algorithm successfully identified individuals not wearing safety helmets in 1109 out of the 1230 videos, achieving an accuracy of 90.2%. The average processing time for skeleton extraction and gait recognition based on the safety helmet detection results was 6.1ms and 0.08ms respectively, meaning an average processing time of 9.18ms per frame, meeting the real-time requirements.

[0137] In summary, the helmet-wearing detection method based on multi-scale features of this invention solves the problem of target scale transformation at construction sites and meets real-time requirements. The proposed data augmentation method shows that the balance of sample classes can be disregarded in some cases. Transfer learning based on the CSPDarkNet53 model proves that pre-trained models on large datasets such as MS COCO can be transferred to helmet-wearing detection, improving the algorithm's performance, alleviating the problem of insufficient data, and avoiding the disadvantage of needing to label a large amount of dataset. The gait recognition designed based on the helmet-wearing detection results can effectively identify the identity of people not wearing helmets and achieve timely and accurate reminders.

[0138] The above description is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1.A method for detecting a safety helmet wearing in a complex scene, characterized in that, The method comprises: Step A, according to the characteristics of the large size change of the detected target in the picture under the complex scene, four characteristic scales of safety helmet detection are designed; On the basis of YOLO v5 network detecting neck and head, a fourth detection scale with smaller receptive field is added to enhance the detection effect of small targets, and the last four detection scales are 13*13, 26*26, 52*52 and 104*104, which have larger scale detection range than the original three scales; When YOLO v5 is trained, the objective function of the bounding box regression is related to the true value B gt and the predicted value B, which is calculated as shown in equation (1). where d is the center C of the true value gt and the distance between the center c of the predicted box, 1 is the diagonal length of the minimum circumscribed rectangle of the true value and the predicted box, and IoU is the intersection over union of the two rectangular boxes, which is calculated as shown in equation (2). Alpha is a weight parameter, and its definition is shown in formula (3), v is the aspect ratio consistency, according to the true value and the width w of the prediction box gt , w and the height h gt , h are calculated as shown in equation (4), According to the loss function calculation formula, it can be determined that generating more suitable prediction boxes can improve the training effect of the network; Step B, the attention mechanism is introduced into the YOLO v5 backbone network, and the improved YOLO v5 network is designed to reduce the loss of low-level effective information extraction to high-level information; The introduction of the fourth detection scale will lead to the deepening of the network layers, and the loss will occur when the low-level information is transmitted to the high-level information, so the attention mechanism is introduced to reduce the information loss and improve the transmission efficiency, and at the same time, the network can pay attention to small targets in the image; For a convolutional mapping F tr : X→ U, X∈R H′×W′×C′ , U∈R H×W×C , v c represents the cth convolutional kernel, then the output U = [u1, u2,..., u c ] is given by equation (5), u c = v c *X (5) The introduced attention mechanism includes squeezing operation and excitation operation, the squeezing operation encodes the spatial features of one channel into a global feature, and the global average pooling is adopted, as shown in formula (6), And the excitation operation adopts the sigmoid (sigma) activation function as shown in formula (7), s = F ex (z, W) = σ(g(z, W)) = σ(W2δ(W1z)) (7) where δ is a RELU activation function, r is the dimension reduction ratio; finally, the learned channel weights are assigned to the original features to obtain the final output as shown in equation (8). X=s*u (8) The attention mechanism makes the model pay more attention to the channel features with the largest amount of information and suppresses the unimportant channel features, and two SE modules are introduced, which are introduced before the second and third convolution splicing operations in the backbone network; Step C, from the aspects of precision and speed, the evaluation index is designed to evaluate the detection effect, and the data augmentation and CutMix data enhancement are carried out for the categories with poor detection effect; In order to compare the improvement brought by the improved model, the improved model and the model before improvement are trained and tested on the same training set and test set, and are evaluated from the aspects of precision and speed, the precision and recall are calculated, and the PR curve is drawn, the mean average precision (MAP) is calculated, and the calculation formulas are shown in formulas (9), (10) and (11), where TP is the number of targets correctly predicted by the model, FN is the number of other objects wrongly predicted as targets, FN is the number of true values that are not correctly predicted by the model; n is the number of classes, and AP is the area under the corresponding PR curve for each class i The average detection time on the same test set is used as an indicator in terms of speed. After the evaluation is completed, targeted data augmentation and CutMix data enhancement are performed on the categories with poor detection effect. Let x ∈ R W×H×C and y represent the training target and its label, respectively. The CutMix data augmentation operation for A and B two images is shown in equation (12) and equation (13). where M ∈ {0, 1} W×H is a mask matrix marked by 0 and 1, which is consistent with the image size, 0 is the region that needs to be cropped, and 1 is the region that needs to be retained; is element-wise multiplication, and λ is a uniform distribution subject to (0, 1); The data enhancement operation can make the model focus on the features that are difficult to distinguish, and improve the training efficiency; Step D, based on the improved YOLO v5, the feature extraction ability of the CSPDarkNet53 model of the backbone network is transferred, and the safety helmet recognition in the complex scene is realized; After the improved model can achieve good results on the safety helmet detection data set, the model of other similar detection tasks can further improve the generalization ability of the safety helmet detection model, The generalization of transfer learning is defined as follows: Conditions: Given a source domain D s and a learning task T on the source domain s , a target domain D t and a learning task T on the target domain t Objective: To utilize D s and T s learning a prediction function f(·) over the target domain Restriction: D s ≠ D t , T s ≠ T t The backbone network CSPDarkNet53 of the transfer learning is preliminarily trained on a large dataset such as MS COCO and completes a corresponding detection task, all convolutional layers and pre-classification fully connected layers of the backbone network are frozen, and are transplanted into an improved model, and are retrained on the enhanced dataset in step C to obtain a safety helmet wearing detection model using multi-scale features, the safety helmet wearing state of workers is detected on a construction site, and a detection result is output, the result has two categories, a person without wearing a safety helmet and a hat; Step E, based on the detection result of the improved YOLO v5, a gait recognition algorithm module is designed to realize identity recognition of the person without wearing a safety helmet: In order to reduce the amount of calculation, only the person class detection result is subjected to gait recognition, for the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2) of the detection frame, the width and height of the head can be calculated to infer the human body boundary frame of the person without wearing a safety helmet, as shown in formulas (14)-(19), h = y2-y1 (14) w = x2-x1 (15) x3 = max(0, x1-2×w) (16) y3 = min(0, y1-h) (17) x4 = min(width, x2+2×w) (18) y4 = min(height, y2+5×h) (19) Wherein width and height are the width and height of the picture, (x3, y3), (x4, y4) are the upper left corner and lower right corner coordinates of the human body boundary frame; 17 skeleton key points are extracted in the boundary frame by using HRNet, and a skeleton key point gait sequence composed of the skeleton key points is input into a gait recognition network GaitGraph for recognition. The skeleton of a human body can be represented structurally as G = (V, ε), where V = {v1,..., v n} is a set of N nodes representing the joints, and ε is a set of edges of the skeleton represented by an adjacency matrix A ∈ R N×N , when v i is connected to v j , A i，j = 1, otherwise A j，j = 0, since G is non-directed, A is a symmetric matrix; Human gait can be characterized as X = {x} t，n ∈R c |t, n∈Z, 1≤t≤T, 1≤n≤N}, for a feature tensor X∈R T×N×C , where x t，n =X t，n For node v n In the C-dimensional feature vector at time t, there are a total of T frames, X t ∈R N×C At time t, the pose feature X in dimension C represents the corresponding 2D coordinates and its confidence level, and N is the number of nodes; Thus, the learnable weight matrix of the network l-th layer can be represented as which at time t can be used for the feature's graph convolutional neural network layer update computation as follows: wherein is the skeleton graph with the identity matrix added to guarantee the node's own characteristics, is a diagonal matrix, and σ(·) is an activation function. The gait recognition network is composed of a ResGCN module, the module is composed of a graph convolution, a common two-dimensional time domain convolution and a residual connection with a bottleneck structure; the whole network is composed of multiple ResGCN modules arranged in sequence, the output layer is composed of an average pooling layer and a fully connected layer generating a feature vector, a supervised contrast loss is used as a loss function, and the network finally outputs a code corresponding to a recognition target and a corresponding feature vector.

Images