Safety helmet detection model quantification method for edge intelligent equipment

Abstract

The invention provides a safety helmet detection model quantification method for edge intelligent equipment. Through data acquisition, data enhancement and construction of a target detection model combining YOLOv5 and an SE attention mechanism, the safety helmet detection accuracy can be remarkably improved, meanwhile, model quantization is performed by using an MLIR deep learning compiler, memory occupation and calculation requirements are greatly reduced, and the detection model can efficiently run on edge equipment with limited resources. The method can be used for efficiently detecting the wearing condition of the safety helmet in scenes such as construction sites and workshops, can also greatly reduce hardware cost and energy consumption, and has wide application prospects and market value.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and target detection Internet of Things, and specifically relates to a helmet detection model quantization method for edge intelligent devices. Background Art

[0002] With the rapid development of deep learning technology, neural network models have been widely adopted in many industries and application scenarios. However, these models are large in scale and require a lot of hardware storage and computing resources. Especially in the field of object detection for edge computing smart devices, this high requirement conflicts with the limited hardware resources. The scale of model parameters often limits the deployable hardware devices. To overcome this challenge, model quantization technology has emerged. Model quantization technology significantly reduces the memory usage and computing requirements of the model by reducing model parameters from high-precision values ​​(such as Float32) to low-precision values ​​(such as INT8). This process includes integer quantization, dynamic range quantization, and mixed precision quantization methods to ensure the applicability and performance of the model in different scenarios. Model quantization not only enables the model to run efficiently on low-power devices, but also significantly reduces hardware costs and energy consumption, and improves the deployment capabilities of edge computing devices.

[0003] Multi-Level Intermediate Representation (MLIR) is a compiler infrastructure developed by Google to address the limitations of traditional compilers in optimization and code generation. The design of MLIR is inspired by the need to support deep learning and machine learning models on multiple hardware and software platforms. The principle is to gradually convert high-level neural network models into low-level code suitable for different hardware through hierarchical intermediate representations. The MLIR framework includes multiple levels of intermediate representations, from high-level operators to low-level hardware instructions, and each layer allows specific optimizations and conversions. Through this multi-level representation and optimization mechanism, MLIR can effectively optimize the computing performance and resource utilization of the model, and support the efficient deployment of complex neural network models on various hardware platforms. Currently, pre-trained neural networks (TORCH, ONNX, Caffe and other models) under different frameworks can be converted into model files that can be efficiently operated on edge smart devices through specialized MLIR quantization tools. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a method for quantizing a helmet detection model for edge intelligent devices. Through data collection, data enhancement, and the construction of a target detection model combining YOLOv5 and SE attention mechanism, the accuracy of helmet detection can be significantly improved. At the same time, the MLIR deep learning compiler is used for model quantization, which greatly reduces memory usage and computing requirements, so that the detection model can run efficiently on resource-constrained edge devices. The present invention can not only be used for efficient detection of helmet wearing conditions in scenes such as construction sites and workshops, but also can greatly reduce hardware costs and energy consumption, and has broad application prospects and market value.

[0005] A helmet detection model quantification method for edge intelligent devices, characterized by the following steps:

[0006] Step 1, obtain a helmet image dataset: collect images containing people wearing helmets to form a helmet image dataset;

[0007] Step 2, data enhancement and labeling: perform data enhancement processing on the images in the data set, and then label the enhanced images to obtain labels. The labeling content includes whether a helmet is worn and the target frame position of the helmet; the labels are stored as a label file in txt format;

[0008] Step 3, build a helmet target detection network model: use the YOLOv5 model as the basic detection model, modify the original yolov5s.yaml configuration file, add the SE module before the SPPF module of its backbone network Backbone, and obtain the helmet target detection network model;

[0009] Step 4, model training: using the image dataset processed in step 2 as input, train the helmet target detection network model based on the PyTorch deep learning framework to obtain the trained detection model helmet.pt;

[0010] Step 5, model quantization: Use the MLIR deep learning compiler to quantize the detection model trained in step 4 to obtain an INT8 symmetric quantization model file;

[0011] Step 6, model deployment and application: Load and run the INT8 symmetric quantization model in step 5 on the edge computing device, and verify and apply the model using the video image to be detected.

[0012] Specifically, images containing people wearing helmets are collected from the Internet and public image and video datasets.

[0013] Specifically, the data enhancement processing includes image cropping, rotation, brightness transformation and saturation transformation processing.

[0014] Specifically, the labelimg annotation software is used to annotate the images.

[0015] Specifically, the YOLOv5 model mainly includes a backbone network Backbone, a neck network Neck and a prediction module Prediction, wherein the backbone network Backbone includes a standard convolutional layer, a CSP module and an SSPF module, and uses a deep convolutional neural network to extract features from the input image; the neck network Neck integrates multi-scale features from the backbone network Backbone and performs feature fusion; the prediction module Prediction performs target detection on the feature map processed by the neck network Neck, and outputs the target bounding box and category probability.

[0016] Specifically, the SE module introduces an attention mechanism in the channel dimension, and improves the weight of important feature channels by adaptively weighting the feature map of each channel. The specific processing process is: perform feature transformation F on the input feature map X tr (·), generate the intermediate feature map U; then, through the compression operation F sp (·) Global information aggregation is performed on the intermediate feature map U to generate a channel descriptor; then, after the excitation operation F sp (·; W) generates a channel weight vector s and passes through the scaling operation F scale (·) Adjust each channel of the original feature map channel by channel, and finally output the recalibrated feature map The size of the feature map X is H′×W′×C′, where H′, W′, and C′ represent the height, width, and number of channels of the feature map, respectively. The size of is H×W×C, where H, W, and C represent the height, width, and number of channels of the feature map, respectively.

[0017] Specifically, when training the helmet target detection network model, the stochastic gradient descent method is used for model optimization, the initial learning rate learning-rate = 0.01, the hyperparameter weight_decay = 0.0005, and the number of training iterations are set to 289 times.

[0018] Specifically, the specific process of step 5 is:

[0019] Step a: Modify the return statement in the forward function in model / yolo.py, and convert the PT detection model helmet.pt into an ONNX model helmet.onnx with three outputs through the export.py file. The three outputs are [1,3,80,80,7], [1,3,40,40,7], and [1,3,20,20,7], where 3 represents 3 anchor boxes and 7 refers to 5+2, where 2 is the number of categories, each category number corresponds to a label, and 5 represents the four coordinates of the box plus a confidence level; the feature map resolutions of the three output layers are 80*80, 40*40, and 20*20, respectively;

[0020] Step b: In the ONNX-MLIR toolchain, use onnx_2_mlir.py to convert the ONNX model helmet.onnx to mlir format, compile the model, set the input size to [1,3,640,640], the mean of each channel of the image, the default is mean[0.0,0.0,0.0], and keep the original aspect ratio of the image. The output file is yolov5_helmet.mlir;

[0021] Step c: Use the model_calibration.py file to perform model calibration by using data training data cleaning to generate the calibration file yolov5-se-calibration;

[0022] Step d: Take yolov5_helmet.mlir and yolov5-se-calibration as input, and compile the yolo_complie.py file to generate the INT8 symmetric quantization model;

[0023] Step e: Write the detect_yolov5_se.py test file based on the helmet target detection network model, verify the detection accuracy of the quantization model for the helmet target, obtain the image frames with detection boxes generated by the pt model prediction and the image frames with detection boxes generated by the quantization model prediction, compare the detection confidence of the same image frame, and obtain the accuracy difference between the pt model and the quantization model.

[0024] Specifically, the specific process of model calibration by using data training data cleaning is as follows: 100-200 images are randomly selected from the original training data set of the model to form a calibration data set, the images in the calibration data set are input into the detection model, the minimum and maximum values ​​of the output of each layer of the network in the statistical model are obtained, and these statistical values ​​are used to determine the quantization scaling factor and zero point, and the floating point weights and activation values ​​are quantized to obtain the quantized model.

[0025] Specifically, the generation of the INT8 symmetric quantization model refers to mapping the model floating-point number range to the signed 8-bit integer range [-128, 127] through a unified scaling factor, wherein the quantization range is the same in the positive and negative directions, and the specific steps are as follows:

[0026] Step (1): Calculate the minimum value x of the data to be quantized min and the maximum value x max ;

[0027] Step (2): Calculate the scale factor scale using the maximum absolute value of the data range:

[0028]

[0029] Step (3): Quantify, according to Convert the floating point value x to INT8 integer value q x , where q x ∈[-128, 127];

[0030] Step (4): Dequantization, according to Set the integer value q of IN8 x Convert back to floating point value

[0031] The beneficial effects of the present invention are as follows: by adopting the YOLOv5+SE network model, the detection accuracy is significantly improved; due to the use of the MLIR deep learning compiler for model quantization, the memory usage and computing requirements are greatly reduced, so that the detection model can run efficiently on resource-constrained edge devices; the present invention utilizes the entire process from data collection, data enhancement, model training to quantization deployment, has high compatibility and flexible deployment capabilities, can adapt to a variety of edge intelligent hardware devices, and significantly reduce hardware costs and energy consumption; comprehensive model accuracy verification ensures the high detection rate and reliability of the quantization model in practical applications, and provides an efficient, energy-saving and easy-to-deploy solution for the safety helmet detection of edge intelligent devices, which has broad application prospects and market value; through the model quantization method of the present invention, safety helmet detection can be efficiently carried out in scenarios such as construction sites and manufacturing workshops, improving the level of safety management, reducing the occurrence of safety accidents, and further promoting the application and development of edge computing technology in the field of industrial safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of a quantification method of a helmet detection model for edge intelligent devices of the present invention;

[0033] Figure 2 It is the SE module structure diagram of the present invention;

[0034] Figure 3This is an example diagram of the effect of helmet detection using the method of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the accompanying drawings and embodiments. The present invention includes but is not limited to the following embodiments.

[0036] The present invention provides a helmet detection model quantization method for edge intelligent devices, such as Figure 1 As shown, the specific implementation process is as follows:

[0037] Step 1, obtain a helmet image dataset: collect images of people wearing helmets from the Internet and public image and video datasets (such as video surveillance datasets of production lines and warehouse scenes) to form a helmet image dataset.

[0038] Step 2, data enhancement and labeling: Perform data enhancement processing on the images in the data set to obtain an enhanced helmet data set to be tested. The data enhancement processing includes image cropping, rotation, brightness transformation and saturation transformation. Then, label the enhanced images, such as using labelimg labeling software to label the images to obtain labels. The labeling content includes whether a helmet is worn and the target frame position of the helmet. The labels are stored in a label file in txt format. The label structure can be expressed as:

[0039] txt i ={class i , x i ,y i , w i ,h i} (2)

[0040] Among them, txt i is the annotation box information of the i-th image in the dataset, class i is the category code: "Not worn-0", "Weared-1"; x i is the horizontal coordinate of the center of the annotation box, y i is the center ordinate of the annotation box, w i is the width of the annotation box, h i The height of the label box.

[0041] Step 3, build a hard hat target detection network model: use the YOLOv5 model as the basic detection model, modify the original yolov5s.yaml configuration file, add the SE module ([-1, 1, SE,

[1024] ]) before the SPPF module of its backbone network Backbone, and obtain the hard hat target detection network model.

[0042] The YOLOv5 model structure mainly includes the backbone network Backbone, the neck network Neck and the prediction module Prediction. The backbone network Backbone uses a deep convolutional neural network to extract features from the input image. The neck network Neck integrates multi-scale features from the backbone network Backbone and performs feature fusion. The prediction module Prediction performs target detection on the feature map processed by the neck network Neck and outputs the target bounding box and category probability. The backbone network Backbone includes a standard convolution layer (Conv), a CSP module (CSP Bottleneck) and a spatial pyramid pooling module (Spatial Pyramid Pooling-Fast, SSPF). After the SE module (Squeeze-and-Excitation) is introduced, they work together to extract, compress and enhance the feature information of the input image layer by layer.

[0043] The SE module introduces an attention mechanism in the channel dimension, and by adaptively weighting the feature map of each channel, it increases the weight of important feature channels, thereby enhancing the effective features in the target detection task and improving the accuracy of target detection. Figure 2 As shown in the figure, the SE attention mechanism transforms the input feature map X by F tr (·), generate the intermediate feature map U. Then, through the compression operation F sp (·) Global information aggregation is performed on U to generate a channel descriptor. Then, after the excitation operation F sp (·; W) generates a channel weight vector s and passes through the scaling operation F scale (·) Adjust each channel of the original feature map channel by channel, and finally output the recalibrated feature map This mechanism aims to enhance useful features and suppress irrelevant information, thereby improving the expressiveness of the model. The size of the feature map X is H′×W′×C′, where H′, W′, and C′ represent the height, width, and number of channels of the feature map respectively; the feature maps U and The size is H×W×C, where H, W, and C represent the height, width, and number of channels of the feature map, respectively.

[0044] Step 4, model training: The network training is based on the PyTorch deep learning framework, implemented in Python 3.8, and runs in CUDA 11.5. The hardware environment for model training is a server equipped with a 24GB NVIDIA RTX 4090 graphics card. The software environment for model training is the Ubuntu 20.04 operating system. The model is optimized using the stochastic gradient descent method, with the initial learning rate learning-rate = 0.01 and the hyperparameter weight_decay = 0.0005. The model training converged after 289 generations, generating the best training model best_helmet.pt.

[0045] Step 5, model quantization: Use the MLIR deep learning compiler to quantize the detection model trained in step 4 to obtain an INT8 symmetric quantization model file, which specifically includes the following steps:

[0046] Step a: Modify the return statement in the forward function in model / yolo.py, and convert the PT detection model helmet.pt into an ONNX model helmet.onnx with three outputs through the export.py file. The three outputs are [1,3,80,80,7], [1,3,40,40,7], and [1,3,20,20,7]. 3 represents 3 anchor boxes, 7 refers to 5+2, where 2 is the number of categories, each category number corresponds to a label, and 5 represents the four coordinates of the box plus a confidence level; the feature map resolutions of the three output layers are 80*80, 40*40, and 20*20, respectively.

[0047] Step b: Model compilation. In the ONNX-MLIR toolchain, use onnx_2_mlir.py to convert the ONNX model helmet.onnx to mlir format;

[0048] Step c: mlir model calibration. Use the model_calibration.py file to perform model calibration by using data training data cleaning to generate the calibration file yolov5-se-calibration. The specific process is:

[0049] 100-200 images are randomly selected from the original training dataset of the model to form a calibration dataset. After forward propagation of the calibration dataset (i.e., the images in the calibration dataset are input into the detection model), the minimum and maximum outputs of each layer in the network are counted, and these statistical values ​​are used to determine the quantization scaling factor and zero point, and the floating-point weights and activation values ​​are quantized. Finally, a quantized model is generated to improve the inference efficiency of the model on edge devices.

[0050] Step d: Convert the MLIR intermediate file to the INT8 model. Take yolov5_helmet.mlir and yolov5-se-calibration as input, and compile the yolo_complie.py file to generate the INT8 symmetric quantization model.

[0051] Generating an INT8 symmetric quantization model means mapping the model floating-point range to a signed 8-bit integer range [-128, 127] through a unified scale factor, where the quantization range is the same in the positive and negative directions. Specifically, this quantization method determines the quantization scale factor by calculating the maximum value max and minimum value min of the floating-point number, so that the floating-point range [min, max] is linearly mapped to the integer range [-128, 127]. The specific steps are as follows:

[0052] 1): Calculate the minimum value x of the data to be quantized min and the maximum value x max ;

[0053] 2): Calculate the scale factor scale using the maximum absolute value of the data range:

[0054]

[0055] 3): Quantization, converting the data floating point value x into INT8 integer value q x , where q x ∈[-128,127]:

[0056]

[0057] 4): Dequantize IN8 integer value q x Convert back to floating point value

[0058]

[0059] Step e: Write the detect_yolov5_se.py test file based on the helmet target detection network model, verify the detection accuracy of the quantization model for the helmet target, and compare it with the original PyTorch model, that is, obtain the image frame with detection box predicted by the pt model and the image frame with detection box predicted by the quantization model, compare the detection confidence of the same image frame, and obtain the accuracy difference between the pt model and the quantization model.

[0060] Step 6, model deployment and application: After the quantized and optimized model is deployed to the computing device, the optimized network can be loaded and run on the edge computing device to build a helmet object detection system. The reliability and performance of the quantized model on the edge device are ensured by testing and verification in the actual environment, thereby verifying its effectiveness and accuracy in actual applications.

[0061] Figure 3 Three exemplary pictures of the effect of helmet detection using the method of the present invention are given. In the picture, the person above the detection box is the detection category, and the number is the confidence (i.e. the category probability belonging to the category).

[0062] The present invention provides a quantization method for a helmet detection model for edge intelligent devices, which aims to solve the problem of limited resources when deep learning models are deployed on edge computing devices. The method adopts a target detection model that combines YOLOv5 with the SE attention mechanism, and uses the MLIR deep learning compiler to quantize the model, converting the high-precision weights and calculations of the model into a low-precision integer format (such as INT8). This quantization significantly reduces the memory usage and computing requirements of the model, so that the detection model can run efficiently even on resource-constrained edge devices. The overall method not only realizes efficient detection of helmet wearing in scenes such as construction sites and workshops, but also greatly reduces hardware costs and energy consumption, and has broad application prospects and market value. In addition, while ensuring a high detection rate, the method also ensures the characteristics of low energy consumption and easy deployment, and is particularly suitable for scenes such as construction sites and manufacturing workshops that require helmet detection.

Claims

1. A quantification method for helmet detection model for edge intelligent devices, characterized in that Here are the steps: Step 1, obtain a helmet image dataset: collect images containing people wearing helmets to form a helmet image dataset; Step 2, data enhancement and labeling: perform data enhancement processing on the images in the data set, and then label the enhanced images to obtain labels. The labeling content includes whether a helmet is worn and the target frame position of the helmet; the labels are stored as a label file in txt format; Step 3, build a helmet target detection network model: use the YOLOv5 model as the basic detection model, modify the original yolov5s.yaml configuration file, add the SE module before the SPPF module of its backbone network Backbone, and obtain the helmet target detection network model; Step 4, model training: using the image dataset processed in step 2 as input, train the helmet target detection network model based on the PyTorch deep learning framework to obtain the trained detection model helmet.pt; Step 5: Model quantization: Use the MLIR deep learning compiler to quantize the detection model trained in step 4 to obtain an INT8 symmetric quantization model file; Step 6, model deployment and application: Load and run the INT8 symmetric quantization model in step 5 on the edge computing device, and verify and apply the model using the video image to be detected.

2. A helmet detection model quantification method for edge intelligent devices according to claim 1, characterized in that: Images containing people wearing helmets are collected from the Internet and public image and video datasets.

3. The method for quantifying a helmet detection model for edge intelligent devices according to claim 1, characterized in that: The data enhancement processing includes image cropping, rotation, brightness transformation and saturation transformation processing.

4. The method for quantifying a helmet detection model for edge intelligent devices according to claim 1, characterized in that: Use labelimg annotation software to annotate images.

5. The method for quantifying a helmet detection model for edge intelligent devices according to claim 1, characterized in that: The YOLOv5 model mainly includes a backbone network Backbone, a neck network Neck and a prediction module Prediction, wherein the backbone network Backbone includes a standard convolutional layer, a CSP module and an SSPF module, and uses a deep convolutional neural network to extract features from the input image; the neck network Neck integrates multi-scale features from the backbone network Backbone and performs feature fusion; the prediction module Prediction performs target detection on the feature map processed by the neck network Neck, and outputs the target bounding box and category probability.

6. The method for quantifying a helmet detection model for edge intelligent devices according to claim 1, characterized in that: The SE module introduces an attention mechanism in the channel dimension, and improves the weight of important feature channels by adaptively weighting the feature map of each channel. The specific processing process is: perform feature transformation F on the input feature map X tr (·), generate the intermediate feature map U; then, through the compression operation F sp (·) Global information aggregation is performed on the intermediate feature map U to generate a channel descriptor; then, after the excitation operation F sp (·; W) generates a channel weight vector s and passes through the scaling operation F scale (·) Adjust each channel of the original feature map channel by channel, and finally output the recalibrated feature map The size of the feature map X is H′×W′×C′, where H′, W′, and C′ represent the height, width, and number of channels of the feature map, respectively. The size of is H×W×C, where H, W, and C represent the height, width, and number of channels of the feature map, respectively.

7. The method for quantifying a helmet detection model for edge intelligent devices according to claim 1, characterized in that: When training the helmet target detection network model, the stochastic gradient descent method is used for model optimization, the initial learning rate learning-rate = 0.01, the hyperparameter weight_decay = 0.0005, and the number of training iterations are set to 289 times.

8. The method for quantifying a helmet detection model for edge intelligent devices according to claim 1, characterized in that: The specific process of step 5 is: Step a: Modify the return statement in the forward function in model / yolo.py, and convert the PT detection model helmet.pt into an ONNX model helmet.onnx with three outputs through the export.py file. The three outputs are [1,3,80,80,7], [1,3,40,40,7], and [1,3,20,20,7], where 3 represents 3 anchor boxes and 7 refers to 5+2, where 2 is the number of categories, each category number corresponds to a label, and 5 represents the four coordinates of the box plus a confidence level; the feature map resolutions of the three output layers are 80*80, 40*40, and 20*20, respectively; Step b: In the ONNX-MLIR toolchain, use onnx_2_mlir.py to convert the ONNX model helmet.onnx to mlir format, compile the model, set the input size to [1,3,640,640], the mean of each channel of the image, the default is mean[0.0,0.0,0.0], and keep the original aspect ratio of the image. The output file is yolov5_helmet.mlir; Step c: Use the model_calibration.py file to calibrate the model by using data training data cleaning to generate the calibration file yolov5-se-calibration; Step d: Take yolov5_helmet.mlir and yolov5-se-calibration as input, and compile the yolo_complie.py file to generate the INT8 symmetric quantization model; Step e: Write the detect_yolov5_se.py test file based on the helmet target detection network model, verify the detection accuracy of the quantization model for the helmet target, obtain the image frames with detection boxes generated by the pt model prediction and the image frames with detection boxes generated by the quantization model prediction, compare the detection confidence of the same image frame, and obtain the accuracy difference between the pt model and the quantization model.

9. A helmet detection model quantification method for edge intelligent devices according to claim 8, characterized in that: The specific process of model calibration by using data training data cleaning is as follows: 100-200 images are randomly selected from the original training data set of the model to form a calibration data set, the images in the calibration data set are input into the detection model, the minimum and maximum values ​​of the output of each layer of the network in the statistical model are obtained, and these statistical values ​​are used to determine the quantization scaling factor and zero point, and the floating point weights and activation values ​​are quantized to obtain the quantized model.

10. The method for quantifying a helmet detection model for edge intelligent devices according to claim 8, characterized in that: The generation of the INT8 symmetric quantization model refers to mapping the model floating point number range to the signed 8-bit integer range [-128, 127] through a unified scaling factor, wherein the quantization range is the same in the positive and negative directions. The specific steps are as follows: Step (1): Calculate the minimum value x of the data to be quantized min and the maximum value x max ; Step (2): Calculate the scale factor scale using the maximum absolute value of the data range: Step (3): Quantify, according to Convert the floating point value x to INT8 integer value q x , where q x ∈[-128, 127]; Step (4): Dequantization, according to Set the integer value q of IN8 x Convert back to floating point value

Images