Safety helmet detection method, model training method, system, device and storage medium

Abstract

The embodiment of the application provides a safety helmet detection method, a model training method, a system, a device and a storage medium, and belongs to the technical field of artificial intelligence. The second image feature is obtained by inputting the training data into the backbone network layer of the model for feature extraction and then connecting the network layer for feature integration. The input feature is subjected to average pooling, high-dimensional convolution and global average pooling operations in parallel in the connection network layer, and then the feature fusion is performed to obtain the output feature. Thus, the important local information and global context information in the image can be focused on, the loss of safety helmet feature information is reduced, the safety helmet detection related features in the comprehensive training data are learned, the second image feature is input into the head network layer of the model for feature prediction to obtain a predicted detection frame, and the safety helmet detection model is reversely updated based on the predicted detection frame and the real detection frame. The safety helmet detection model trained by the application has high robustness, and can accurately detect the safety helmet wearing condition for a blurred monitoring picture.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a safety helmet detection method, a model training method, a system, a device and a storage medium. BACKGROUND

[0002] It is well known that the wearing of safety helmets is crucial on construction sites, but there are still people with a lucky mentality on construction sites who think that wearing safety helmets is too troublesome and do not wear safety helmets. There are many construction site practitioners, and the danger is high, and accidents are prone to occur. At present, management personnel mainly judge whether safety helmets are worn by manually checking on-site monitoring videos, but this method requires a lot of manpower and material resources, and is also prone to omissions due to personal negligence.

[0003] In recent years, with the development of computer vision technology, the detection of scene objects in images and videos has become a research hotspot. The target detection model can identify multiple information in an image and judge the category and position of the object. The basis of target detection is the convolutional neural network (CNN), which has been successfully used in target detection, target segmentation and target recognition tasks based on image data. The convolutional layer is the cornerstone of CNN, which is mainly responsible for extracting useful feature information of the input image, and using these feature information can perform subsequent target object detection work. Using a target detection algorithm to detect the safety helmets of workers in the on-site monitoring picture, due to the low resolution of the monitoring picture, the movement of the workers, and the small proportion of the safety helmet picture, the detection target picture in the monitoring is blurred, thereby affecting the accuracy of the safety helmet wearing detection. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide a safety helmet detection method, a model training method, a system, a device and a storage medium, which aims to improve the accuracy of safety helmet wearing detection.

[0005] To achieve the above purpose, the first aspect of the embodiments of the present application provides a safety helmet detection model training method, comprising the following steps:

[0006] Obtain training data and initialize a safety helmet detection model;

[0007] Input the training data into the backbone network layer of the safety helmet detection model for feature extraction to obtain first image features;

[0008] Input the first image features into the connection network layer of the safety helmet detection model for feature integration to obtain second image features, wherein the connection network layer is used for performing feature fusion on the first image features after parallel average pooling, high-dimensional convolution and global average pooling operations to obtain the second image features.

[0009] inputting the second image feature into a head network layer of the safety helmet detection model for feature prediction to obtain a predicted detection frame;

[0010] determining a prediction loss according to the predicted detection frame and a real detection frame in the training data;

[0011] updating parameters of the safety helmet detection model according to the prediction loss to obtain a trained safety helmet detection model.

[0012] In some embodiments, the training data is obtained by:

[0013] obtaining a plurality of monitoring picture images;

[0014] labeling a real detection frame according to a safety helmet in the monitoring picture image to obtain a visual object class format label;

[0015] converting the visual object class format labels of the plurality of monitoring picture images into a text file and converting the plurality of monitoring picture images into a preset specification to obtain the training data.

[0016] In some embodiments, the safety helmet detection model is initialized by:

[0017] extracting a first size of all real detection frames from the training data;

[0018] performing clustering operation on the first size of all real detection frames according to a preset anchor frame number by using a clustering algorithm to obtain a clustering cluster corresponding to the preset anchor frame number, and taking a clustering center of each clustering cluster as a second size of an initial anchor frame;

[0019] constructing a plurality of auxiliary detection frames according to positions of the real detection frames of each monitoring picture image in the training data, and training the safety helmet detection model by taking the auxiliary detection frames as auxiliary labels.

[0020] In some embodiments, the first image feature is input into a connection network layer of the safety helmet detection model for feature integration to obtain a second image feature, by:

[0021] inputting the first image feature into a first branch unit of the connection network layer for average pooling and linear convolution operation to obtain a first tensor;

[0022] inputting the first image feature into a second branch unit of the connection network layer for high-dimensional convolution operation to obtain a second tensor;

[0023] The first image feature is input into a third branch unit of the connection network layer for global average pooling and linear convolution operation, to obtain a third tensor;

[0024] The first tensor, the second tensor and the third tensor are subjected to feature fusion operation to obtain a second image feature.

[0025] In some embodiments, the feature fusion operation on the first tensor, the second tensor and the third tensor to obtain the second image feature comprises the following steps:

[0026] The first tensor and the second tensor are added to obtain a fourth tensor;

[0027] The fourth tensor and the third tensor are multiplied to obtain the second image feature.

[0028] In some embodiments, the determination of the prediction loss according to the predicted detection frame and the real detection frame in the training data comprises the following steps:

[0029] A classification loss is determined according to a predicted classification result of the predicted detection frame and a real classification result;

[0030] A positioning loss is determined according to a position of the predicted detection frame and a position of the real detection frame;

[0031] A confidence loss is determined according to a shape similarity of the predicted detection frame and the real detection frame;

[0032] The prediction loss is determined according to the classification loss, the positioning loss and the confidence loss.

[0033] To achieve the above object, a second aspect of the embodiment of the present application proposes a safety helmet detection method, comprising the following steps:

[0034] A monitoring picture is acquired;

[0035] The safety helmet detection model trained by the first aspect is used to analyze the monitoring picture frame by frame to obtain a safety helmet wearing detection result.

[0036] To achieve the above object, a third aspect of the embodiment of the present application proposes a safety helmet detection model training system, comprising:

[0037] A first module is configured to acquire training data and initialize a safety helmet detection model;

[0038] A second module is configured to input the training data into a backbone network layer of the safety helmet detection model for feature extraction to obtain a first image feature;

[0039] The third module is configured to input the first image feature into a connection network layer of the safety helmet detection model to perform feature integration, and obtain a second image feature.

[0040] The fourth module is configured to input the second image feature into a head network layer of the safety helmet detection model to perform feature prediction, and obtain a predicted detection frame.

[0041] The fifth module is configured to determine a prediction loss according to the predicted detection frame and a real detection frame in the training data.

[0042] The sixth module is configured to update parameters of the safety helmet detection model according to the prediction loss, and obtain a trained safety helmet detection model.

[0043] To achieve the above object, a fourth aspect of the embodiment of the present application provides an electronic device, which comprises a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory, and the program is executed by the processor to realize the method in the first aspect or the second aspect.

[0044] To achieve the above object, a fifth aspect of the embodiment of the present application provides a storage medium, which is a computer readable storage medium, for computer readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the method in the first aspect or the second aspect.

[0045] The safety helmet detection method, model training method, system, device and storage medium provided by the present application can input training data into a backbone network layer of a safety helmet detection model to perform feature extraction to obtain a first image feature, input the first image feature into a connection network layer of the safety helmet detection model to perform feature integration to obtain a second image feature, perform feature fusion on the first image feature in parallel in the connection network layer after average pooling, high-dimensional convolution and global average pooling operations, output the second image feature, pay attention to important local information and global context information in an image, reduce loss of safety helmet feature information, learn safety helmet detection related features in comprehensive training data, input the second image feature into a head network layer of the safety helmet detection model to perform feature prediction to obtain a predicted detection frame, and update the safety helmet detection model in reverse based on the predicted detection frame and a real detection frame. The safety helmet detection model trained by the present application has high robustness, and can accurately detect safety helmet wearing conditions for blurred monitoring pictures. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a safety helmet detection model training method flowchart provided by an embodiment of the present application.

[0047] Figure 2 is Figure 1 is a flowchart of step S101 in

[0048] Figure 3 is Figure 1 is a flowchart of step S101 of acquiring training data in

[0049] Figure 4 is Figure 1 is a flowchart of step S103 of initializing a safety helmet detection model in

[0050] Figure 5 is Figure 4 is a flowchart of step S404 in

[0051] Figure 6 is Figure 1 is a flowchart of step S105 in

[0052] Figure 7 is a safety helmet detection method flowchart provided by an embodiment of the present application.

[0053] Figure 8 is a safety helmet detection model training system schematic diagram provided by an embodiment of the present application.

[0054] Figure 9 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application.

[0055] Figure 10 is a connection network layer schematic diagram provided by an embodiment of the present application.

[0056] Figure 11 is a loss function-based bounding box regression schematic diagram provided by an embodiment of the present application.

[0057] Figure 12 is a safety helmet detection process schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0059] It should be noted that although the functional modules are divided in the device schematic diagram, the logical order is shown in the flowchart, but in some cases, the steps shown or described can be performed in a manner different from the module division in the device, or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to limit the present application.

[0061] First, the meanings of several terms involved in the present application are analyzed:

[0062] Artificial intelligence (AI): is a new technical science to study, develop the theory, method, technology and application system for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science, artificial intelligence attempts to understand the essence of intelligence, and produces a new intelligent machine that can react in a similar way to human intelligence, including robots, language recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, to perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] Convolutional neural network, convolutional neural network (CNN): a class of feedforward neural networks containing convolutional computation and having deep structure, which can be supervised learning through labeled training data, so as to complete visual image recognition and target detection tasks.

[0064] YOLO model (You Only Look Once: Unified, Real-Time Object Detection) is a deep learning algorithm in computer vision technology that can be used for image recognition. YOLO model converts the target detection problem into a Regression regression class problem, that is, given an input image, directly regress the target bounding box and its classification category on multiple positions of the image. YOLO model includes but is not limited to Yolov3, Yolov4, Yolov5 (all are different versions of YOLO) and other models. Different models have different weights, network structure diagrams and algorithms, and different region sampling methods are used.

[0065] IoU is a standard for measuring the accuracy of detecting corresponding objects in a specific dataset. IoU is a simple measurement standard that can be used to measure the task of deriving a target detection range (bounding boxex) in the output. IoU is used to measure the correlation between the true and predicted values, and the higher the correlation, the higher the value. IoU is the result of dividing the overlapping part of two regions by the set part of the two regions.

[0066] Anchor Box algorithm, Anchor Boxes or Prior Boxes. Anchor box is a pre-defined rectangular box with different sizes and aspect ratios, usually uniformly distributed at different positions and scales in the image. The key of anchor box algorithm is to capture targets of different sizes and shapes by generating anchor boxes at different scales and positions. By predicting and screening anchor boxes, the detected target box and its corresponding class probability can be obtained. The application of anchor box algorithm in YOLOv4 model can realize fast and accurate detection and positioning of multiple targets in the image.

[0067] Clustering algorithm, clustering is a very important learning method in data mining, which refers to grouping similar unlabeled sample data into the same class. Clustering belongs to unsupervised learning problem, its goal is to divide the sample set into multiple classes, ensuring that the samples in the same class are as similar as possible, and the samples in different classes are as different as possible. These classes are also called clusters.

[0068] The Kmeans algorithm is a clustering algorithm. Taking binary classification as an example, two sample points can be randomly selected as initial centroids from all sample points to be classified, the distance between each sample point and the two initial centroids is calculated, and the sample points closer to an initial centroid are classified into the class to which the initial centroid belongs, so as to complete the first clustering. After the first clustering is completed, two classes are obtained, the center coordinates of each class are calculated, and the center coordinates of each class are used as new centroids to perform the foregoing classification process according to the distance, so as to obtain the second clustering result. The same iteration process as the foregoing process is performed until the centroids of the two classes of the clustering result no longer change, and the two classes obtained at this time are the final clustering result.

[0069] Embodiments of the present application provide a safety helmet detection method, a model training method, a system, a device, and a storage medium, aiming to improve the accuracy of safety helmet detection.

[0070] Embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0071] Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0072] The safety helmet detection model training method or safety helmet detection method provided by the embodiments of the present application relates to the field of artificial intelligence. The safety helmet detection model training method or safety helmet detection method provided by the embodiments of the present application can be applied in a terminal, can also be applied in a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, etc.; the server can be configured as a standalone physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and basic cloud computing services such as big data and artificial intelligence platforms; the software can be an application that implements the safety helmet detection model training method or safety helmet detection method, but is not limited to the above forms.

[0073] The application is operable in a variety of general purpose or special purpose computing system environments or configurations. Examples of computing systems, environments, and / or configurations that can be suitable for use with the application include personal computers, server computers, handheld or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media including memory storage devices.

[0074] Figure 1 is an optional flowchart of the safety helmet detection model training method provided by the embodiment of the application, Figure 1 The method in the embodiment can include, but is not limited to, steps S101 to S106.

[0075] In step S101, training data is obtained, and a safety helmet detection model is initialized.

[0076] In step S102, the training data is input into a backbone network layer of the safety helmet detection model for feature extraction to obtain first image features.

[0077] In step S103, the first image features are input into a connection network layer of the safety helmet detection model for feature integration to obtain second image features, wherein the connection network layer is used to perform feature fusion on the first image features after performing average pooling, high-dimensional convolution and global average pooling operations in parallel to obtain the second image features.

[0078] In step S104, the second image features are input into a head network layer of the safety helmet detection model for feature prediction to obtain a predicted detection frame.

[0079] In step S105, a prediction loss is determined according to the predicted detection frame and a real detection frame in the training data.

[0080] In step S106, parameters of the safety helmet detection model are updated according to the prediction loss to obtain a trained safety helmet detection model.

[0081] In step S101 of some embodiments, a large number of images of the safety helmet image dataset can be composed by monitoring the scene, or a safety helmet image dataset can be crawled on the network by using python technology, or a larger safety helmet image dataset can be further downloaded by using the kaggle competition open source safety helmet dataset. After obtaining the safety helmet image dataset, the image of the worker in the image can be framed and labeled by using artificial labeling, and the image label is obtained by labeling whether the safety helmet is worn, and the image label is saved in VOC format to form sample data. Further, the sample data can be divided into a training set, a validation set and a test set, the training set is used to build a model and learn image features, the validation set is used to adjust the hyperparameters of the model and to preliminarily evaluate the ability of the model, and the test set is used to evaluate the generalization ability of the final model, but cannot be used as the basis for algorithm-related selection such as parameter adjustment and feature selection. The division ratio of the training set, the validation set and the test set can be 7:1:2. Initializing the safety helmet detection model includes but is not limited to initializing anchor box specifications, model parameters, etc.

[0082] In step S102 of some embodiments, the safety helmet detection model is a target detection model based on a convolutional neural network architecture, and in this embodiment, the safety helmet detection model can adopt a Yolov4 model architecture. The safety helmet detection model is divided into three parts: a backbone network layer (Backbone), a connection network layer (Neck) and a head network layer (Head). Backbone is the main component of the model, usually a convolutional neural network (CNN) or a residual neural network (ResNet), etc., responsible for extracting features of the input image for subsequent processing and analysis, Backbone has many layers and many parameters, and can extract high-level feature representations of the image. Neck is the intermediate layer connecting Backbone and Head, and the main function of Head is to reduce the dimension or adjust the features from Backbone to better adapt to the task requirements, Neck includes convolution, pooling operations, etc. Head is the last layer of the model, usually a classifier or regressor, which produces the final output by inputting the features processed by Head, and Head can be a bounding box regressor and a classifier, etc.

[0083] In step S103 of some embodiments, the connection network layer adopts a parallel attention module (PAM), that is, the connection network layer performs feature fusion on the first image features output by the backbone network layer based on parallel average pooling, high-dimensional convolution, and global average pooling operations to obtain second image features. For example, the PAM is introduced into the backbone network CSPDarknet53 of Yolov4 in the embodiments of the present application. The parallel attention module can utilize features to minimize the loss of feature information and improve the accuracy of predicting small targets and the robustness of the network. The average pooling operation can mine the main feature information of the first image features, reduce the redundant information in the spatial dimension, and highlight the features that are more strongly associated with the detection target. The high-dimensional convolution operation has more weights to adjust at each position, can capture a larger receptive field, understand the context information of the input, and extract rich feature representations. The global average pooling operation can combine feature extraction and nonlinear mapping to improve the accuracy of the model, suppress the overfitting phenomenon in the network, map multiple features to a low-dimensional space, and improve the generalization ability of the model.

[0084] In step S104 of some embodiments, the second image features output by the connection network layer are input into the head network layer of the safety hat detection model for target detection and classification prediction to obtain a predicted detection frame. The result of the predicted detection frame includes human target detection and whether the human wears a safety hat.

[0085] In step S105 of some embodiments, a prediction loss is determined according to the predicted detection frame of the image and the real detection frame in the training data. The prediction loss can be calculated by a loss function. In machine learning, a loss function is used for parameteric estimation of a model. The loss value obtained based on the loss function can be used to describe the difference between the predicted value and the actual value of the model. Common loss functions include mean square error loss function, support vector machine (SVM) hinge loss function, cross-entropy loss function, and the like.

[0086] In step S106 of some embodiments, the propagation of data in the safety helmet detection model has two ways, one is along the path from input to output, called forward propagation, the other is from output back to input, called back propagation. In the process of forward propagation, the monitoring picture image is processed layer by layer through the convolutional neural network and transmitted to the output layer, the error between the predicted output and the real expectation is described through the loss function, and the back propagation is turned into, the partial derivative of the loss function to each neuron weight is calculated layer by layer, the weight gradient data of the loss function to the weight vector is constructed, which is used as the basis for updating the weight parameters, and the training of the safety helmet detection model is completed in the process of continuously updating the weight parameters. Illustratively, the safety helmet detection model adopts Yolov4 model architecture, the safety helmet detection model outputs the predicted detection frame based on the initial anchor frame, compares the predicted detection frame and the real frame, calculates the difference between the two, and then uses gradient back propagation to update the parameters in the network through multiple iterations. The model is evaluated and verified by test data set and verification data, so as to obtain the trained safety helmet detection model. In the training process, 50 times of frozen training and 50 times of unfreezing training can be used, the batch size of frozen training is 32, and the initial learning rate is 0.001.

[0087] Please refer to Figure 2 The step of obtaining training data in step S101 can include, but is not limited to, steps S201 to S203:

[0088] Step S201, obtaining a plurality of monitoring picture images;

[0089] Step S202, marking a real detection frame according to the safety helmet in the monitoring picture image, and obtaining a visual object class format label;

[0090] Step S203, converting the visual object class format label of the plurality of monitoring picture images into a text file, and converting the plurality of monitoring picture images into a preset specification to obtain training data.

[0091] In this embodiment, after obtaining a large number of monitoring picture images, a person marks a real detection frame according to a safety helmet feature in the monitoring picture image to obtain a Visual Object Classes (VOC) format label of a visual object class. VOC is a commonly used labeling format of a target detection and image segmentation dataset. The VOC format is originally introduced by a PASCAL VOC (Visual Object Classes) project and is used for holding a target detection and image segmentation competition. A dataset in the VOC format usually includes an image file (.jpg,.png, etc.) and a labeling file (.xml). The image file includes image data to be labeled, and the labeling file includes labeling information of a target related to each image. Each labeling file corresponds to one image, and includes information such as a category of the target, coordinates of a bounding box, and a segmentation mask of the target.

[0092] Further, the visual object class format labels of the plurality of monitoring picture images are converted into text files in a YOLO format label txt file by using a script tool. Specifically, left upper corner coordinates, right lower corner coordinates, and width and height data information of an image labeling frame in the VOC format label are processed and converted into center coordinates, width, and height data of the labeling frame. Each image in the dataset is set to the same input specification, for example, 640x640, by using a scaling method, so as to obtain training data.

[0093] In some other embodiments, on the basis of obtaining a real detection frame label of training data, a plurality of auxiliary frames similar in shape and size to a real target frame but slightly different in position can be generated by random offsetting near the real detection frame of the image. The auxiliary frame can cover more feature areas, so that the model can better learn the feature representation of the target and improve the accuracy and recall rate of detection. In addition, by introducing the auxiliary frame with random offset, the adaptability of the model to targets of different scales, shapes, and positions can be improved. In a real monitoring picture, a detected human body can appear in different postures and positions. By introducing a plurality of auxiliary frames after offsetting, the generalization ability of the model can be improved, so as to better cope with the target detection task under various conditions.

[0094] Specifically, 16 auxiliary frames are constructed by random offsetting near each real frame, and the auxiliary frames are not limited and can be generated at any position of the image. The auxiliary frame generation process is represented as:

[0095]

[0096] wherein w represents the image width and h represents the image height.

[0097] Please refer to Figure 3The step of initializing the safety helmet detection model in step S101 can include, but is not limited to, steps S301 to S304.

[0098] Step S301: Extracting first sizes of all real detection boxes from training data.

[0099] Step S302: Using a clustering algorithm to cluster the first sizes of all real detection boxes according to a preset anchor box number, to obtain clustering clusters corresponding to the preset anchor box number, and taking a clustering center of each clustering cluster as a second size of an initial anchor box.

[0100] Step S303: Constructing a plurality of auxiliary detection boxes according to positions of real detection boxes of each monitoring picture image in the training data, to train the safety helmet detection model by taking the auxiliary detection boxes as auxiliary labels.

[0101] In some embodiments, the anchor box in the Yolov4 model refers to a candidate box used to predefine different scales and aspect ratios in a target detection task. The anchor box is generated by performing grid division on an input image and setting different scales and aspect ratios. Each feature map unit corresponds to a region on the original image. In order to detect targets of different sizes, we need to set anchor boxes of different aspect ratios and scales for each feature map unit. By applying anchor boxes of different scales to each unit of the feature map, targets of different sizes can be captured. The anchor boxes in the feature map can be used to predict whether there is a target object in each anchor box, and to classify and locate the target. The use of anchor boxes can make the safety helmet detection model have better adaptability in target detection tasks of different scales, thereby improving the detection accuracy and robustness of the algorithm.

[0102] In the present embodiment, a clustering algorithm can be used to initialize the anchor box, so as to determine appropriate anchor box sizes according to the target size distribution of the training data. The clustering algorithm can use K-Medians, K-Means, Hierarchical Clustering, etc., and the present embodiment does not make specific limitations. The real detection box size (height and width) clustering process in the training data can be: extracting the sizes (width and height) of all target boxes from the training data, which can be calculated according to the coordinates of the target box; applying a clustering algorithm such as k-means clustering to cluster the target sizes, the goal of clustering is to divide the first sizes of all real detection boxes in the training data into clustering clusters corresponding to the preset anchor box number, each clustering cluster corresponds to an anchor box; according to the clustering result, taking the clustering center of each cluster as the second size of the anchor box, the second size of the clustering center is subsequently used as the initial anchor box size in the model.

[0103] Please refer to Figure 4, step S103 can include but is not limited to including steps S401 to S404:

[0104] Step S401, input the first image feature into the first branch unit of the connection network layer for average pooling and linear convolution operation to obtain the first tensor;

[0105] Step S402, input the first image feature into the second branch unit of the connection network layer for high-dimensional convolution operation to obtain the second tensor;

[0106] Step S403, input the first image feature into the third branch unit of the connection network layer for global average pooling and linear convolution operation to obtain the third tensor;

[0107] Step S404, perform feature fusion operation on the first tensor, the second tensor and the third tensor to obtain the second image feature.

[0108] In the embodiment, with reference to Figure 10 , the first image feature X extracted by the backbone network layer is R C×H×W , C is the number of image channels, H is the image height, and W is the image width. The first image feature is transmitted in parallel to the three branches of the PAM, i.e. the first branch unit, the second branch unit and the third branch unit.

[0109] In the first branch unit, the first image feature X is sequentially subjected to average pooling and linear convolution (i.e. 1x1 convolution) operation to obtain the first tensor is represented as follows:

[0110]

[0111] Wherein, f avg represents the average pooling operation, conv 1*1 represents the 1x1 convolution operation, which is used to increase the feature map dimension, and outputs the first tensor The first branch unit can be represented by the channel weight.

[0112] In the second branch unit, the first image feature X is subjected to convolution down-sampling feature integration branch, and high-dimensional convolution (such as 3x3 convolution) is used to extract the target feature to obtain the second tensor is represented as follows:

[0113]

[0114] Wherein, conv 3*3 represents the 3x3 convolution operation.

[0115] In the third branch unit, the first image feature X is upgraded in dimension through linear convolution (i.e. 1*1 convolution) after global average pooling operation, and then a third tensor is obtained through a sigmoid activation function is represented as follows:

[0116]

[0117] wherein, represents global average pooling, conv 1*1 represents 1*1 convolution operation, and sigmoid is an activation function.

[0118] Referring to Figure 5 , step S404 can include but is not limited to steps S501 to S502:

[0119] Step S501, adding the first tensor and the second tensor to obtain a fourth tensor;

[0120] Step S502, multiplying the fourth tensor and the third tensor to obtain the second image feature.

[0121] In the embodiment, first, the second tensor output by the second branch unit is added element by element with the first tensor output by the first branch unit to obtain a fourth tensor , thereby reducing the loss of image features, which is represented as follows:

[0122]

[0123] Then, the third tensor output by the third branch unit is multiplied with the fourth tensor to obtain the second image feature X out , which is represented as follows:

[0124]

[0125] Referring to Figure 6 , step S105 includes but is not limited to steps S601 to S604:

[0126] Step S601, determining a classification loss according to a predicted classification result of a predicted bounding box and a real classification result;

[0127] Step S602, determining a positioning loss according to a position of the predicted bounding box and a position of a real bounding box;

[0128] Step S603, determining a confidence loss according to a shape similarity of the predicted bounding box and the real bounding box;

[0129] Step S604, determine the prediction loss according to the classification loss, the positioning loss and the confidence loss.

[0130] In this embodiment, the loss function of the safety helmet detection model has three parts: classification loss, positioning loss and confidence loss. The classification loss refers to the loss of the predicted classification of the model for whether the worker wears a safety helmet and the true classification; the confidence loss refers to the shape similarity of the predicted detection frame output by the model and the true detection frame; and the positioning loss refers to the positioning gap between the predicted detection frame and the true detection frame. In the model training stage, the classification loss, the positioning loss and the confidence loss are combined for use, so as to optimize the performance of the model.

[0131] Specifically, the backbone network layer extracts features, which are input to the head network layer after passing through the network connection layer, to obtain a predicted detection frame through feature prediction. The loss function is used for calculating the positioning and confidence loss of the predicted detection frame, combined with Figure 11 , the model parameter is updated through the loss function, so that the detection frame regression becomes more stable, and the loss function L CloU is as follows:

[0132]

[0133] wherein, IoU represents the correlation between the true detection frame and the predicted detection frame, b represents the center of the predicted detection frame, b gt represents the center of the true detection frame, p represents the Euclidean distance, c represents the diagonal length of the circumscribed rectangle of the predicted detection frame and the true detection frame, a represents a positive compromise parameter, which is generally set to 1, and v represents the consistency of the aspect ratio.

[0134] Further, the specific expressions of a and v are as follows:

[0135]

[0136]

[0137] wherein, w gt is the width of the true detection frame, h gt is the height of the true detection frame, w is the width of the predicted detection frame, and h is the height of the predicted detection frame.

[0138] In this embodiment, the CIOU loss function is used to accelerate the inference speed of the model, improve the learning effect of the features, and reduce the training time of the model.

[0139] Please refer to Figure 7 , the safety helmet detection method provided in the embodiments of the present application includes but is not limited to the following steps S710 to S720:

[0140] Step S710, acquire the monitoring picture;

[0141] Step S720, using the safety helmet detection model trained in the above embodiment to analyze the monitoring picture frame by frame to obtain the safety helmet wearing detection result.

[0142] In the embodiment, please refer to Figure 12 The safety helmet data set marked by the artificial is input into the safety helmet detection model based on the improved Yolov4 architecture to train the safety helmet detection model to reach the preset detection accuracy or reach the training times, and a trained safety helmet detection model is obtained. The monitoring picture video stream collected by the on-site camera is transmitted to the trained safety helmet detection model on the server in real time for frame-by-frame detection analysis to obtain the safety helmet wearing detection result. The safety helmet wearing detection result includes the detection frame of the human body region in the monitoring picture and the confidence score of the detection frame. The confidence score is used to represent the probability of predicting that the human body wears a safety helmet. If the confidence score is greater than 0.5, it is considered that the detection target has worn a safety helmet; if the confidence score is not greater than 0.5, it is considered that the detection target does not wear a safety helmet. In the embodiment, the parallel attention mechanism of the safety helmet detection model can reduce the image feature loss, so that the head network layer can integrate more rich information for prediction, and the safety helmet detection model has strong robustness and can accurately detect whether the object wears a safety helmet in the case that the target object in the monitoring picture is blurred.

[0143] Further, the server can also be linked with the on-site device or other devices according to the detection result, for example, when the detection result is that the worker wears a safety helmet, the on-site gate is driven to open, and at the same time the voice module prompts that the detection is passed successfully; when the detection result is that the worker does not wear a safety helmet, a voice alarm is issued.

[0144] Please refer to Figure 8 The safety helmet detection model training system provided in the embodiment of the application comprises:

[0145] The first module is configured to acquire training data and initialize the safety helmet detection model;

[0146] The second module is configured to input the training data into the backbone network layer of the safety helmet detection model to extract features to obtain first image features;

[0147] The third module is configured to input the first image features into the connection network layer of the safety helmet detection model to integrate features to obtain second image features, wherein the connection network layer is configured to perform feature fusion on the first image features after performing average pooling, high-dimensional convolution and global average pooling operations in parallel to obtain the second image features;

[0148] The fourth module is configured to input the second image feature into a head network layer of the safety helmet detection model to perform feature prediction and obtain a predicted detection frame.

[0149] The fifth module is configured to determine a prediction loss according to the predicted detection frame and a real detection frame in training data.

[0150] The sixth module is configured to update parameters of the safety helmet detection model according to the prediction loss to obtain a trained safety helmet detection model.

[0151] It can be understood that the content in the safety helmet detection model training method embodiments described above is applicable to the system embodiments, the system embodiments specifically implement the same functions as the safety helmet detection model training method embodiments described above, and achieve the same beneficial effects as the safety helmet detection model training method embodiments described above.

[0152] The embodiments of the present application further provide an electronic device, which comprises a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the safety helmet detection model training method or the safety helmet detection method. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0153] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, which comprises:

[0154] The processor 901 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., and is used to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0155] The memory 902 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 902 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 902 and are called and executed by the processor 901 to implement the safety helmet detection model training method or the safety helmet detection method of the embodiments of the present application.

[0156] The input / output interface 903 is used to realize information input and output.

[0157] The communication interface 904 is configured to realize the communication interaction between the device and other devices, and can realize the communication through a wired manner (for example, a USB, a network cable, and the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, and the like).

[0158] The bus 905 is configured to transmit information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0159] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize the communication connection between the device.

[0160] The embodiment of the present application further provides a storage medium, the storage medium is a computer readable storage medium, and the storage medium is used for computer readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the safety helmet detection model training method or the safety helmet detection method.

[0161] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0162] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0163] Those skilled in the art can understand that, Figures 1-6 The technical solutions shown in the above description do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps, or combine certain steps, or different steps.

[0164] The apparatus embodiments described above are merely exemplary, and units described as separate components may or may not be physically separate, i.e., may be located in one place, or may be distributed over multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

[0165] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0166] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims of the foregoing drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so

[0167] It should be understood that in this application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are only A, only B, and A and B at the same time. Where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0168] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0169] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0170] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0171] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0172] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for training a safety helmet detection model, characterized in that, The method comprises the following steps: obtaining training data and initializing a safety helmet detection model; inputting the training data into a backbone network layer of the safety helmet detection model for feature extraction to obtain first image features; inputting the first image features into a connection network layer of the safety helmet detection model for feature integration to obtain second image features, wherein the connection network layer is used for performing feature fusion on the first image features after parallel average pooling, high-dimensional convolution and global average pooling operations to obtain the second image features; inputting the second image features into a head network layer of the safety helmet detection model for feature prediction to obtain a predicted detection frame; determining a prediction loss according to the predicted detection frame and a real detection frame in the training data; updating parameters of the safety helmet detection model according to the prediction loss to obtain a trained safety helmet detection model; the connection network layer specifically performs feature fusion through the following steps: adding a first tensor based on an average pooling operation and a second tensor based on a high-dimensional convolution operation to obtain a fourth tensor; multiplying the fourth tensor and a third tensor based on a global average pooling operation to obtain the second image features. 2.The safety hat detection model training method of claim 1, wherein, The method for obtaining training data comprises the following steps: obtaining multiple monitoring picture images; labeling real detection frames according to safety helmets in the monitoring picture images to obtain visual object class format labels; converting the visual object class format labels of the multiple monitoring picture images into text files and converting the multiple monitoring picture images into a preset specification to obtain training data. 3.The safety hat detection model training method of claim 2, wherein, The method for initializing the safety helmet detection model comprises the following steps: extracting first sizes of all real detection frames from the training data; using a clustering algorithm to perform clustering operations on the first sizes of all real detection frames according to a preset anchor frame number to obtain clustering clusters corresponding to the preset anchor frame number, and taking clustering centers of each clustering cluster as second sizes of initial anchor frames; constructing multiple auxiliary detection frames according to positions of real detection frames of each monitoring picture image in the training data to train the safety helmet detection model by taking the auxiliary detection frames as auxiliary labels. 4.The safety hat detection model training method of claim 3, wherein, The method for inputting the first image features into the connection network layer of the safety helmet detection model for feature integration to obtain the second image features comprises the following steps: inputting the first image features into a first branch unit of the connection network layer for average pooling and linear convolution operations to obtain a first tensor; inputting the first image features into a second branch unit of the connection network layer for high-dimensional convolution operations to obtain a second tensor; inputting the first image features into a third branch unit of the connection network layer for global average pooling and linear convolution operations to obtain a third tensor; performing feature fusion operations on the first tensor, the second tensor and the third tensor to obtain the second image features. 5.The safety hat detection model training method of claim 1, wherein, The method for determining a prediction loss according to the predicted detection frame and the real detection frame in the training data comprises the following steps: determining a classification loss according to a predicted classification result of the predicted detection frame and a real classification result; determining a positioning loss according to a position of the predicted detection frame and a position of the real detection frame; determine a confidence loss according to a shape similarity of the predicted bounding box and the real bounding box; determine a prediction loss according to the classification loss, the positioning loss and the confidence loss.

6. A safety hat detection method characterized by, The method comprises the following steps: obtaining a monitoring picture; performing frame-by-frame analysis on the monitoring picture by using the trained safety helmet detection model according to any one of claims 1 to 5 to obtain a safety helmet wearing detection result. 7.A safety helmet detection model training system, characterized in that, The method comprises: a first module configured to obtain training data and initialize a safety helmet detection model; a second module configured to input the training data into a backbone network layer of the safety helmet detection model to perform feature extraction and obtain first image features; a third module configured to input the first image features into a connection network layer of the safety helmet detection model to perform feature integration and obtain second image features, wherein the connection network layer is configured to perform feature fusion on the first image features after performing average pooling, high-dimensional convolution and global average pooling operations in parallel to obtain the second image features; a fourth module configured to input the second image features into a head network layer of the safety helmet detection model to perform feature prediction and obtain a predicted bounding box; a fifth module configured to determine a prediction loss according to the predicted bounding box and a real bounding box in the training data; a sixth module configured to update parameters of the safety helmet detection model according to the prediction loss to obtain a trained safety helmet detection model; The connection network layer performs feature fusion through the following steps: adding a first tensor obtained based on an average pooling operation and a second tensor obtained based on a high-dimensional convolution operation to obtain a fourth tensor; multiplying the fourth tensor and a third tensor obtained based on a global average pooling operation to obtain the second image features.

8. An electronic device, comprising: The electronic device comprises a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory, and the program, when executed by the processor, realizes the steps of the safety helmet detection model training method according to any one of claims 1 to 5 or the safety helmet detection method according to claim 6.

9. A storage medium, the storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs executable by one or more processors to realize the steps of the safety helmet detection model training method according to any one of claims 1 to 5 or the safety helmet detection method according to claim 6.

Images