Model training method and helmet wearing detection method based on helmet wearing detection

Abstract

The application discloses a model training method and a helmet wearing detection method based on helmet wearing detection, relates to the technical field of target detection, and comprises the following steps: target marking is performed on a helmet sample image; the helmet sample image after marking is input into a feature extraction network to crop a region containing a cyclist's shoulder, head and helmet, so that a head helmet feature map is obtained; then, SPPNet network and PANet network are used to extract context features and perform feature fusion; finally, the output of the PANet network is input into a prediction network to perform helmet position prediction. In each iteration, a loss function is optimized and updated by using an optimization improvement controller, and the optimization improvement controller uses a learning rate reduction method to make the loss function tend to converge. By cropping the target marking, the detection speed is improved, and the learning rate reduction method is used to improve the detection accuracy of the helmet wearing detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, in particular to a helmet wearing detection based model training method and a helmet wearing detection method. BACKGROUND

[0002] According to statistics, the proportion of death caused by brain injury in electric bicycle accidents is as high as 80%, and wearing a helmet can reduce the risk of brain injury to a certain extent. However, using artificial methods to monitor the wearing of electric bicycle helmets will occupy a large amount of labor resources and is inefficient, and although using a camera for remote indirect observation will also have a large amount of work. Therefore, the method of using computers to replace artificial methods has been gradually applied to various aspects of traffic safety.

[0003] At present, there are few methods for detecting the helmets of electric bicycle riders in the method for detecting electric bicycles, and most of the existing detection technologies have problems such as low detection accuracy and slow detection speed. SUMMARY

[0004] The purpose of the present application is to provide a helmet wearing detection based model training method and a helmet wearing detection method with fast detection speed and high accuracy.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] A YOLOv4 model training method based on helmet wearing detection, comprising:

[0007] Step 1: target labeling is performed on a helmet sample image; the helmet sample image is an image of an electric bicycle rider wearing a helmet, and the target includes the rider's shoulder, head and helmet;

[0008] Step 2: input the labeled helmet sample image into a feature extraction network, and the feature extraction network cuts the area containing the rider's shoulder, head and helmet according to the label to obtain a head helmet feature map;

[0009] Step 3: input the head helmet feature map into an SPPNet network to extract context features;

[0010] Step 4: input the output of the SPPNet network into a PANet network for feature fusion;

[0011] Step 5: Input the output of the PANet network into the prediction network to predict the helmet position, and calculate the loss value using a loss function based on the predicted helmet position and the target marker in the helmet sample image. Adjust the weight parameters in the network based on the loss value. In each iteration, the loss function is optimized and updated using an optimization and improvement controller, which uses a learning rate descent method to make the loss function converge.

[0012] Optionally, step 3 specifically includes:

[0013] Multiple convolutions are used to batch normalize the head helmet feature map;

[0014] Multiple max pooling methods of different scales were used to extract contextual features from the batch-normalized head helmet feature maps.

[0015] Optionally, step 4 specifically includes:

[0016] The PANet network is used to fuse parameters of the input feature map through convolution, upsampling, feature fusion, and stacking.

[0017] Optionally, the pooling kernels for max pooling include 13x13, 9x9, 5x5, and 1x1.

[0018] Optionally, the loss function is based on the formula calculate;

[0019] Where b is the parameter representing the predicted center coordinates of the head helmet feature map, b gt ρ represents the parameter of the true target boundary center in the head helmet feature map. 2 The predicted center point is the square of the distance from the center point of the actual target boundary, where c is the length of the diagonal of the minimum bounding rectangle of the two head helmet feature maps. w and h are the height and width of the predicted bounding box of the head helmet feature map, w gt h gt The height and width of the true target bounding box in the head helmet feature map.

[0020] Optionally, the optimization improves the controller according to formula θ t+1 =θ t -η·▽J(θ t The loss function is then optimized.

[0021] Where, θ t+1 Let η be the parameter for the (t+1)th round, η be the learning rate, and θ be the parameter for the (t+1)th round. t Let J(θ) be the parameter for the t-th round. t ) is the loss function, ▽J(θ) t) represents the gradient.

[0022] Optionally, the optimization controller is an Adam optimizer or an SGD optimizer.

[0023] Optionally, the learning rate reduction method is either the step learning rate reduction method or the cos learning rate reduction method.

[0024] The present invention also provides a helmet wearing detection method, comprising:

[0025] Acquire the image to be detected;

[0026] The image to be detected is input into the trained YOLOv4 model as described in claims 1-8 to perform helmet wearing detection on the image to be detected.

[0027] Optionally, after performing helmet-wearing detection on the image to be detected, the method further includes:

[0028] The output of the YOLOv4 model is filtered by non-maximum suppression (NMS) to obtain the filtered helmet-wearing detection results.

[0029] According to specific embodiments of the present invention, the following technical effects are disclosed: a model training method and a helmet wearing detection method based on helmet wearing detection. The training method includes: firstly, target labeling of helmet sample images; the helmet sample images are images of electric bicycle riders wearing helmets, and the targets include the rider's shoulders, head, and helmet; then, inputting the labeled helmet sample images into a feature extraction network, which crops the region containing the rider's shoulders, head, and helmet according to the labeling to obtain a head-helmet feature map; then, inputting the head-helmet feature map into an SPPNet network for context feature extraction; and inputting the output of the SPPNet network into a PANet network for feature fusion; finally, inputting the output of the PANet network into a prediction network for helmet position prediction, and calculating the loss value using a loss function based on the predicted helmet position and the target labeling of the helmet sample images, and adjusting the weight parameters in the network based on the loss value; wherein, in each iteration, the loss function is optimized and updated using an optimization and improvement controller, which uses a learning rate descent method to make the loss function converge. This invention improves the extraction efficiency of the feature extraction network by cropping the markers, thereby accelerating the detection speed of electric vehicle helmet wearing detection. Furthermore, during the iteration process, the loss function is optimized and updated by the controller, and the learning rate descent method is used to make the loss function converge, so that the loss function reaches the minimum output, thereby improving the detection accuracy of helmet wearing detection. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the YOLOv4 model training method based on helmet wearing detection provided in an embodiment of the present invention;

[0032] Figure 2 This is a diagram of the YOLOv4 algorithm structure provided in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram illustrating the convergence of the algorithm under different learning rates according to an embodiment of the present invention. Figure 3 'a' is a diagram illustrating an excessively small learning rate. Figure 3 b is a diagram illustrating a moderate learning rate. Figure 3 c is a diagram illustrating an excessively high learning rate;

[0034] Figure 4 This is a diagram of the LabelImg software annotation interface provided in an embodiment of the present invention;

[0035] Figure 5 This is a schematic diagram illustrating the detection results of different network models under interference-free conditions provided in an embodiment of the present invention; wherein, Figure 5 'a' represents the interference-free image to be detected. Figure 5 b represents the detection result of the interference-free image to be detected by the SSD network. Figure 5 c represents the detection result of the interference-free image to be detected by the YOLOv4 network. Figure 5 d represents the detection result of the interference-free image to be detected by the optimized YOLOv4 network;

[0036] Figure 6 This is a schematic diagram illustrating the detection results of different network models under interference conditions, provided by an embodiment of the present invention; wherein, Figure 6 A disturbed image to be detected. Figure 6 b represents the detection result of the interfering image to be detected by the S SD network. Figure 6 c represents the detection results of the image to be detected using the YOLOv4 network, which contains interference. Figure 6 d represents the detection result of the disturbed image to be detected by the optimized YOLOv4 network;

[0037] Figure 7 This is a schematic diagram illustrating the results of helmet-wearing detection of electric bicycle riders provided in an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] The purpose of this invention is to provide a model training method and a helmet-wearing detection method based on helmet-wearing detection that has a fast detection speed and high accuracy.

[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] like Figure 1 As shown, a YOLOv4 model training method based on helmet-wearing detection includes:

[0042] Step 1: Mark targets in the helmet sample image; the helmet sample image is an image of an electric bicycle rider wearing a helmet, and the targets include the rider's shoulders and head as well as the helmet.

[0043] Step 2: Input the labeled helmet sample image into the feature extraction network. The feature extraction network crops the area containing the rider's shoulders, head and helmet according to the label to obtain the head and helmet feature map.

[0044] Step 3: Input the head helmet feature map into the SPPNet network to extract contextual features.

[0045] Step 4: Input the output of the SPPNet network into the PANet network for feature fusion.

[0046] Step 5: Input the output of the PANet network into the prediction network for iterative training to obtain the weight parameters and loss function updated in the last iteration; wherein, during the iteration process, the loss function is optimized and updated using an optimization improvement controller, which uses a learning rate descent method to make the loss function converge.

[0047] In some embodiments, step 1 may specifically include:

[0048] Write a Python script to perform web scraping, set corresponding keywords, crawl image search websites such as Baidu, obtain images of electric bicycle riders wearing helmets, and set the image format to .jpg.

[0049] Images are named with numbers to ensure a consistent and unique format. LabelImg software is used to label the cyclist's shoulders, head, and helmet. A 2D bounding box is used to label the helmet-wearing area as "WithHelmet," generating an .xml text file. The label information is then read from the .xml file into a .txt file. Each image and .xml text file corresponds one-to-one, with the .xml file containing the image's annotation information, path, size, location, and category.

[0050] For example, images of electric bicycle riders wearing helmets were obtained through web scraping. This experiment used a total of 5212 images as the test data sample, with a training set to test set ratio of 9:1. The VOC dataset format was used, and the dataset was divided into three parts: train, validation, and test, primarily for training, validation, and testing.

[0051] In some embodiments, step 2 may specifically include:

[0052] like Figure 2 As shown, the labeled helmet sample image is input into the feature extraction network CSPDarknet53 to obtain the head and helmet feature map. The feature extraction network CSPDarknet53 uses target cropping for data enhancement, cropping the region containing the cyclist's shoulders, head, and helmet according to the target labels.

[0053] Specifically, the feature extraction network CSPDarknet53 uses the Mish activation function, with the formula: Mish=x×tanh(ln(1+e x )), where x is the input, tanh is the hyperbolic tangent function, and Mish is a smooth non-monotonic activation function that helps maintain small negative values, thus stabilizing the network gradient flow.

[0054] Specifically, after the feature map is input, the feature extraction network finds the leftmost, rightmost, topmost, and bottommost pixels of the target labeled region to obtain coordinate information, and then crops the target labeled region based on the coordinate information to ensure that the target is not cropped out. Furthermore, since the target's position changes, the position of the bounding box also needs to be updated.

[0055] In some embodiments, steps 3 and 4 may specifically include:

[0056] like Figure 2As shown, the head helmet feature map from the last layer of the feature extraction network CSPDarknet53 in step 2 is input into the SPPNet network. After performing three batch normalized convolutions on the last feature layer of CSPDarknet53, it is processed using four different scales of max pooling. The max pooling kernel sizes are 13x13, 9x9, 5x5, and 1x1, which can greatly increase the receptive field and separate the most significant contextual features. Furthermore, max pooling can preserve the original features while reducing the parameters of the neural network training, thus reducing the training time and further accelerating the helmet wearing detection speed.

[0057] The PANet network is then used for parameter aggregation. It is a recurrent pyramid network structure consisting of convolution, upsampling, feature fusion and stacking.

[0058] In some embodiments, step 5 specifically includes;

[0059] The output value of the PANET network after feature fusion in step 4 is input into the prediction network for iterative training. During the iteration process, the loss function is optimized and updated by using the SGD or Adam optimization controller. By adjusting the loss ratio and using the step and / or cos learning rate descent method, the loss tends to converge, achieving the effect of minimizing the output of the loss function.

[0060] Specifically, the loss function is based on the formula calculate.

[0061] Where b is the parameter representing the predicted center coordinates of the head helmet feature map, b gt ρ represents the parameter of the center of the true target bounding box in the head helmet feature map. 2 Let c be the square of the distance between the two center points, and let c be the length of the diagonal of the minimum bounding rectangle of the two head helmet feature maps. w, h and w gt h gt These represent the height and width of the predicted bounding box and the height and width of the ground truth bounding box, respectively.

[0062] Specifically, the controller is optimized and improved according to the formula θ. t+1 =θ t -η·▽J(θ t The loss function CIOU is then optimized.

[0063] Where, θ t+1 Let η be the parameter for the (t+1)th round, η be the learning rate, and θ be the parameter for the (t+1)th round. t Let J(θ) be the parameter for the t-th round. t ) is the loss function, ▽J(θ) t ) represents the gradient.

[0064] Specifically, the `optimizer_type` parameter sets the optimizer type, with options including Adam and SGD optimizers. When using the Adam optimizer, the maximum learning rate is set to 1e-3, while the SGD optimizer sets it to 1e-2. SGD requires a longer convergence time, thus necessitating a larger training epoch, while Adam can use a relatively smaller total training epoch. Providing an optimizer type selection allows for finding the optimal parameters of the built-in model after training, updating and calculating network parameters that affect model training and output, so that the model output approximates or reaches the optimal value. Figure 3 a, Figure 3 b、 Figure 3 As shown in Figure c, different learning rates affect the convergence of the algorithm. A learning rate that is too small will result in slow convergence but is more likely to enter local minima; a moderate learning rate leads to faster convergence, while a learning rate that is too large makes convergence difficult. The `lr_decay_type` setting determines the learning rate descent method, with options including step and cosine descent. Controlling the learning rate allows the model to achieve better convergence.

[0065] In some embodiments, after obtaining the weight parameters and loss function updated in the last iteration, the method may further include:

[0066] Using the trained weight parameters, the network is initialized and fed with the image to be predicted to obtain the final detection result (detection box). Based on the obtained detection boxes, the model's performance is evaluated through mathematical calculations, and the specific detection accuracy is given.

[0067] The evaluation of model performance involves mathematical calculations, specifically including: comprehensively assessing and analyzing prediction performance by calculating metrics such as AP, recall, FPS, log-average miss rate, and ground-truth, and outputting the accuracy. Accuracy refers to the probability that a sample is correctly predicted as positive within a given sample, and the accuracy P is... TP refers to the number of positive samples that are correctly identified, and FP refers to the number of negative samples that are falsely reported.

[0068] Recall is the probability of correctly predicting a positive sample; it represents the model's ability to find relevant targets. The recall rate R is... TP refers to the number of positive samples that were correctly identified, and FN refers to the number of positive samples that were missed.

[0069] Typically, precision (P) is plotted on the ordinate and recall (R) on the x-axis. Using combinations of different precision and recall points, the resulting curve is called the PR curve, and the area under the curve is the AP (Average Precision). The formula for calculating AP is: AP is the average of N accuracy values. A higher AP value indicates a better object detection model. mAP is the average of the AP values ​​across all classes, with values ​​ranging from [0,1]. It is commonly used to calculate object detection accuracy. The formula for mAP is... That is, to calculate the average of N AP values.

[0070] Prediction speed (FPS) represents the number of frames per second that a model can detect in an image. It is used to evaluate the speed of object detection and can be understood as the refresh rate of the image. FPS is calculated as FPS = 1000 / time.

[0071] The present invention also provides a helmet wearing detection method, comprising:

[0072] Obtain the image to be detected, which is an image containing an electric bicycle rider.

[0073] The image to be detected is input into the trained YOLOv4 model to detect helmet wearing by the electric bicycle rider.

[0074] In the helmet-wearing detection of electric bicycle riders, the trained weight parameters are used. The weight parameters are fed into the initialization network, and then the image to be predicted is input to obtain the final detection result.

[0075] In some embodiments, following helmet-wearing detection of the electric bicycle rider, the method may further include:

[0076] The output of the YOLOv4 model was filtered by non-maximum suppression (NMS) to obtain helmet-wearing detection results that included electric bicycle riders wearing helmets.

[0077] Specifically, two convolution operations are performed using 3×3 and 1×1 kernels. The three predicted bounding boxes generated from each feature layer are then analyzed to determine if they contain the required feature information. Non-maximum suppression (NMS) and prior box adjustments are then applied to obtain the final predicted bounding boxes, which can then be used to detect features such as… Figure 7 The target to be detected can be seen in the image shown.

[0078] In summary, the present invention has the following advantages:

[0079] 1) This invention achieves the effect of minimizing the loss function output by adjusting the loss ratio and using the step and / or cos learning rate descent method to make the loss converge. It also uses SGD and Adam as optimization to improve the controller.

[0080] 2) This invention performs target labeling by adding data augmentation to target cropping, and the CSPDarkNet backbone feature extraction network uses the Mish function as the activation function, achieving a higher efficiency of the feature extraction network.

[0081] 3) The optimized YOLOv4 algorithm provided by this invention has the highest AP value, meaning it achieves the highest accuracy of 84.65%. Recall represents the proportion of predicted results that cover the true targets in the test set. Through training optimization, the recall rate was improved by 2.27 points compared to the original YOLOv4. The detection time is within 0.2 seconds, indicating a fast detection speed. The improved and optimized YOLOv4 algorithm reduced the false positive rate to 0.39, achieving both improved accuracy and a lower false positive rate. Algorithm robustness refers to the model's relatively robust and stable state. While possessing high detection accuracy and precision, small or large errors in the assumptions will not significantly or catastrophically affect the model. In real-life situations, many hats can interfere with the detection results. Even when other hats appear in the detection image, the target group can still be accurately detected.

[0082] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0083] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for training a YOLOv4 model based on helmet-wearing detection, characterized in that, include: Step 1: Mark targets in the helmet sample image; the helmet sample image is an image of an electric bicycle rider wearing a helmet, and the targets include the rider's shoulders, head, and helmet; Step 2: Input the labeled helmet sample image into the feature extraction network. The feature extraction network crops the area containing the rider's shoulders, head and helmet according to the label to obtain the head and helmet feature map. Step 3: Input the head helmet feature map into the SPPNet network to extract contextual features; Step 4: Input the output of the SPPNet network into the PANet network for feature fusion; Step 5: Input the output of the PANet network into the prediction network to predict the helmet position, and calculate the loss value using a loss function based on the predicted helmet position and the target marker in the helmet sample image. Adjust the weight parameters in the network based on the loss value. In each iteration, the loss function is optimized and updated using an optimization and improvement controller, which uses a learning rate descent method to make the loss function converge. The loss function is based on the formula calculate; Where b is the parameter of the predicted center coordinate of the head helmet feature map, bgt is the parameter of the true target boundary center of the head helmet feature map, and ρ 2 The predicted center point is the square of the distance from the center point of the actual target boundary, where c is the length of the diagonal of the minimum bounding rectangle of the two head helmet feature maps. w and h are the height and width of the predicted bounding box of the head helmet feature map, w gt h gt The height and width of the true target bounding box in the head helmet feature map; The optimized controller is based on formula θ t+1 =θ t -η·▽J(θ t The loss function is then optimized. Where, θ t+1 Let η be the parameter for the (t+1)th round, η be the learning rate, and θ be the parameter for the (t+1)th round. t Let J(θ) be the parameter for the t-th round. t ) is the loss function, ▽J(θ) t Let θ be the gradient of the loss function at time t. The feature extraction network CSPDarknet53 uses target cropping for data augmentation, cropping regions containing the cyclist's shoulders, head, and helmet based on target markers. CSPDarknet53 employs the Mish activation function, with the formula: Mish=x×tanh(ln(1+e)). x )), where x is the input, tanh is the hyperbolic tangent function, and Mish is a smooth non-monotonic activation function; after the feature map is input, the feature extraction network obtains the coordinate information by finding the leftmost, rightmost, topmost, and bottommost pixels of the target labeled region, and then crops the target labeled region according to the coordinate information.

2. The YOLOv4 model training method based on helmet-wearing detection according to claim 1, characterized in that, Step 3 specifically includes: Multiple convolutions are used to batch normalize the head helmet feature map; Multiple max pooling methods of different scales were used to extract contextual features from the batch-normalized head helmet feature maps.

3. The YOLOv4 model training method based on helmet-wearing detection according to claim 1, characterized in that, Step 4 specifically includes: The PANet network is used to fuse parameters of the input feature map through convolution, upsampling, feature fusion, and stacking.

4. The YOLOv4 model training method based on helmet-wearing detection according to claim 2, characterized in that, The pooling kernels for maximum pooling include 13x13, 9x9, 5x5, and 1x1.

5. The YOLOv4 model training method based on helmet-wearing detection according to claim 1, characterized in that, The optimization and improvement controller is either the Adam optimizer or the SGD optimizer.

6. The YOLOv4 model training method based on helmet-wearing detection according to claim 1, characterized in that, The learning rate reduction method is either the step learning rate reduction method or the cos learning rate reduction method.

7. A method for detecting helmet wearing, characterized in that, include: Acquire the image to be detected; The image to be detected is input into the trained YOLOv4 model as described in claims 1-6 to perform helmet wearing detection on the image to be detected.

8. The helmet wearing detection method according to claim 7, characterized in that, After performing helmet-wearing detection on the image to be detected, the method further includes: The output of the YOLOv4 model is filtered by non-maximum suppression (NMS) to obtain the filtered helmet-wearing detection results.

Images