Helmet detection method and device based on improved YOLOv5, medium and terminal

Abstract

The invention discloses a helmet detection method and device based on improved YOLOv5, a medium and a terminal, relates to the technical field of computer vision and target detection, and mainly aims at solving the problems that an existing target detection method is low in detection rate, and the accuracy rate is difficult to meet actual requirements. Comprising the steps of collecting a real-time image of a target object based on edge equipment as a target image; performing helmet detection processing on a target object in the target image based on a helmet detection model which is deployed in the edge device and has completed model training, and obtaining a helmet detection result of the target object; and extracting an identification mark of the target object from the target image, performing mapping processing on the identification mark and the helmet detection result, and sending a mapping result to a traffic monitoring database.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and object detection, and particularly relates to a helmet detection method and device based on an improved YOLOv5, a medium and a terminal. BACKGROUND

[0002] With the wide popularity of electric bicycles, motorcycles and other vehicles, the road traffic safety problems caused by them are increasingly prominent. Correctly wearing a helmet is a key measure to protect the life safety of riders. Relying solely on manual patrol and judgment has problems such as low efficiency, high cost, limited coverage, and being easily affected by subjective factors.

[0003] With the development of artificial intelligence technology, target detection algorithms based on deep learning, especially the YOLO series algorithms, have been widely used in helmet detection tasks due to their good balance between speed and accuracy.

[0004] However, on the one hand, the complex structure and large amount of calculation of the backbone network of the YOLOv5 model result in a heavy model, which in turn reduces the target detection rate. On the other hand, the information fusion module of the YOLOv5 model has poor capability, which makes it difficult to meet the demand for the accuracy of the target detection result. SUMMARY

[0005] Therefore, the present application provides a helmet detection method and device based on an improved YOLOv5, a medium and a terminal, which mainly aims to improve the problems of low detection rate and low accuracy of existing target detection methods.

[0006] According to one aspect of the present application, a helmet detection method based on an improved YOLOv5 is provided, which comprises: collecting a real-time image of a target object based on an edge device as a target image; performing helmet detection processing on the target object in the target image based on a helmet detection model trained in the edge device, to obtain a helmet detection result of the target object, wherein the helmet detection model is constructed based on a YOLOv5 neural network as a basic architecture, a lightweight reparameterization module (SimRepCSP) as a feature extraction layer, and a dual convolution feature fusion module (C3_DualConv) as a feature fusion layer; extracting an identification of the target object from the target image, mapping the identification with the helmet detection result, and sending the mapping result to a traffic monitoring database.

[0007] Preferably, the lightweight reparameterization module is used for feature extraction processing, which comprises: inputting the target image into the lightweight reparameterization module; performing multi-level feature transformation processing on the target image by using the plurality of convolution layers and the plurality of reparameterization convolution layers in the lightweight reparameterization module to obtain a multi-level feature transformation processing result; performing fusion processing on the multi-level feature transformation processing result and the target image to obtain a multi-size feature map group, wherein the reparameterization convolution layer is reparameterized into a single convolution kernel in an inference process.

[0008] Preferably, the feature fusion processing is performed by using a double convolution feature fusion module, including: inputting the multi-size feature map group into a first branch and a second branch of the double convolution feature fusion module respectively; performing convolution transformation processing on the multi-size feature map group in the first branch to obtain a first branch output feature map; in the second branch, performing deep feature extraction operations on the multi-size feature map group based on a plurality of BottleNeck modules, wherein in each BottleNeck module, the multi-size feature map group is divided into a preset number of groups, at least one group of the plurality of groups is subjected to parallel convolution operation to obtain a parallel convolution result, the remaining groups are subjected to separate convolution operation to obtain a separate convolution result, and the parallel convolution result and the separate convolution result are added to obtain a second branch output feature map; performing channel dimension splicing processing and integration processing on the first branch output feature map and the second branch output feature map to obtain a multi-size feature map group after feature fusion, and performing position positioning and class prediction on the target object based on the multi-size feature map group after feature fusion.

[0009] Preferably, before the helmet detection model trained based on the edge device detects the target object in the target image to obtain the helmet detection result of the target object, the method further comprises: taking YOLOv5 neural network as a basic architecture, and using a lightweight reparameterization module as a feature extraction layer and a double convolution feature fusion module as a feature fusion layer to construct an initial helmet detection model; obtaining a training sample set, wherein the sample images are labeled with the positions and classes of the detection objects, and the classes include wearing a helmet and not wearing a helmet; performing helmet detection processing on the detection objects in each of the sample images based on the initial helmet detection model to obtain a plurality of initial helmet detection results; calculating a loss function value between the plurality of initial helmet detection results and a plurality of sample image annotation results, wherein the loss function value is composed of an improved bounding box regression loss value, a confidence loss value and a classification loss value, the improved bounding box regression loss value is calculated based on the following formula,

[0010] denotes the improved bounding box regression loss value, L MPDIoU denotes the loss value of the MPDIoU, IoU denotes the intersection over union between the predicted box and the real box, IoU Inner denotes the intersection over union between the Inner predicted box and the Inner real box; based on the loss function value, the model parameters of the initial helmet detection model are iteratively updated by back propagation until a preset iteration stopping condition is reached, obtaining a helmet detection model that has completed model training; deploying the helmet detection model that has completed model training in the edge device to perform helmet detection in the edge device.

[0011] Preferably, the deploying the helmet detection model that has completed model training in the edge device comprises: converting the helmet detection model that has completed model training into an intermediate format helmet detection model; performing model optimization processing and serialization processing on the intermediate format helmet detection model using a model optimizer to obtain an engine file corresponding to the helmet detection model that has completed model training, wherein the model optimization processing includes layer fusion optimization processing, kernel tuning processing and calculation precision adjustment processing; deploying the engine file in the edge device.

[0012] Preferably, before the helmet detection model that has completed model training deployed in the edge device performs helmet detection processing on the target object in the target image to obtain the helmet detection result of the target object, the method further comprises: performing size standardization processing and pixel normalization processing on the target image to obtain a preprocessed target image, so as to perform helmet detection processing on the target object in the preprocessed target image.

[0013] Preferably, before the helmet detection model that has completed model training deployed in the edge device performs helmet detection processing on the target object in the target image to obtain the helmet detection result of the target object, the method further comprises: The target image is subjected to resolution adjustment processing to obtain a target image with completed resolution adjustment, so as to perform helmet detection processing on a target object in the target image with completed resolution adjustment.

[0014] According to another aspect of the present application, an improved YOLOv5-based helmet detection device is provided, comprising: An image acquisition module is configured to acquire a real-time image of a target object based on an edge device as a target image. A helmet detection module is configured to perform helmet detection processing on a target object in the target image based on a helmet detection model with completed model training deployed in the edge device to obtain a helmet detection result of the target object, wherein the helmet detection model is constructed based on a YOLOv5 neural network as a basic architecture, a lightweight reparameterization module (SimRepCSP) as a feature extraction layer, and a dual convolution feature fusion module (C3_DualConv) as a feature fusion layer. A detection result sending module is configured to extract an identification of the target object from the target image, perform mapping processing on the identification and the helmet detection result, and send a mapping result to a traffic monitoring database.

[0015] Preferably, the helmet detection module is configured to: input the target image into the lightweight reparameterization module; perform multi-level feature transformation processing on the target image by using multiple convolution layers and multiple reparameterization convolution layers in the lightweight reparameterization module to obtain a multi-level feature transformation processing result; perform fusion processing on the multi-level feature transformation processing result and the target image to obtain a multi-size feature map group, wherein the reparameterization convolution layer is reparameterized into a single convolution kernel in an inference process.

[0016] Preferably, the helmet detection module is further configured to: input the multi-size feature map group into a first branch and a second branch of the dual convolution feature fusion module, respectively; perform convolution transformation processing on the multi-size feature map group in the first branch to obtain a first branch output feature map; In the second branch, based on a plurality of BottleNeck modules, depth feature extraction operations are sequentially performed on the multi-size feature map groups, wherein in each BottleNeck module, the multi-size feature map groups are divided into a preset number of groups, at least one of the plurality of groups is subjected to parallel convolution operation to obtain a parallel convolution result, the remaining groups are subjected to separate convolution operation to obtain a separate convolution result, and the parallel convolution result and the separate convolution result are added to obtain a second branch output feature map. The first branch output feature map and the second branch output feature map are subjected to channel dimension splicing processing and integration processing to obtain a multi-size feature map group after feature fusion, so as to perform position positioning and category prediction on the target object based on the multi-size feature map group after feature fusion.

[0017] Preferably, before the helmet detection module, the device further comprises a model training module, comprising: A model training unit is configured to use YOLOv5 neural network as a basic architecture, use a lightweight reparameterization module as a feature extraction layer, and use a double convolution feature fusion module as a feature fusion layer to construct an initial helmet detection model; obtain a training sample set, wherein the sample images are labeled with the positions and categories of the detection objects, and the categories include wearing a helmet and not wearing a helmet; perform helmet detection processing on the detection objects in each sample image based on the initial helmet detection model to obtain a plurality of initial helmet detection results; calculate the loss function value between the plurality of initial helmet detection results and a plurality of sample image annotation results, wherein the loss function value is composed of an improved bounding box regression loss value, a confidence loss value, and a classification loss value, and the improved bounding box regression loss value is calculated based on the following formula,

[0018] represents the improved bounding box regression loss value, L MPDIoU represents the loss value of MPDIoU, IoU represents the intersection over union between the predicted box and the real box, IoU Inner represents the intersection over union between the Inner predicted box and the Inner real box; based on the loss function value, the model parameters of the initial helmet detection model are iteratively updated through back propagation until a preset iteration stopping condition is reached to obtain a helmet detection model after model training; A deployment unit is configured to deploy the helmet detection model after model training in the edge device to perform helmet detection in the edge device.

[0019] Preferably, the deployment unit is configured to: convert the completed model training helmet detection model into an intermediate format helmet detection model; perform model optimization processing and serialization processing on the intermediate format helmet detection model using a model optimizer to obtain an engine file corresponding to the completed model training helmet detection model, wherein the model optimization processing includes layer fusion optimization processing, kernel tuning processing, and calculation precision adjustment processing; deploy the engine file in the edge device.

[0020] Preferably, before the helmet detection module, the device further includes an image preprocessing module for: performing size normalization processing and pixel normalization processing on the target image to obtain a preprocessed target image, so as to perform helmet detection processing on the target object in the preprocessed target image.

[0021] Preferably, before the helmet detection module, the device further includes an image resolution adjustment module for: performing resolution adjustment processing on the target image to obtain a resolution-adjusted target image, so as to perform helmet detection processing on the target object in the resolution-adjusted target image.

[0022] According to another aspect of the present application, a storage medium is provided, the storage medium storing at least one executable instruction, the executable instruction causing a processor to perform operations corresponding to the improved YOLOv5-based helmet detection method described above.

[0023] According to still another aspect of the present application, a terminal is provided, comprising a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface completing communication with each other through the communication bus. The memory is used to store at least one executable instruction, the executable instruction causing the processor to perform operations corresponding to the improved YOLOv5-based helmet detection method described above.

[0024] Through the above technical solutions, the technical solutions provided by the embodiments of the present application have at least the following advantages: The application provides a helmet detection method and device based on improved YOLOv5, a medium and a terminal. First, real-time images of a target object are collected based on an edge device as target images. Second, a helmet detection model trained in the edge device is used to detect the target object in the target image to obtain a helmet detection result of the target object. The helmet detection model is based on a YOLOv5 neural network, uses a lightweight reparameterization module (SimRepCSP) as a feature extraction layer, and uses a dual convolution feature fusion module (C3_DualConv) as a feature fusion layer. Finally, an identification of the target object is extracted from the target image, the identification is mapped with the helmet detection result, and the mapping result is sent to a traffic monitoring database. Compared with the prior art, the lightweight reparameterization module is used as the feature extraction layer to simplify the network structure of the traditional YOLOv5 neural network, so that the model parameters are fewer and the calculation amount is lower, thereby improving the target detection rate. Meanwhile, the dual convolution feature fusion module is used as the feature fusion layer to improve the poor information fusion capability of the traditional YOLOv5 neural network, enhance the detection capability of the model on multi-scale targets, and improve the detection precision and accuracy. The above description is only a summary of the technical scheme of the application. In order to more clearly understand the technical means of the application, the application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0025] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included only to illustrate preferred embodiments and are not considered to be limiting of the present application. Moreover, in the entire description and drawings, like reference numerals are used to designate like parts throughout the several views. In the drawings: Figure 1 A helmet detection method flowchart based on improved YOLOv5 is shown; Figure 2 Another helmet detection method flowchart based on improved YOLOv5 is shown; Figure 3 A helmet detection model training flowchart is shown; Figure 4 A lightweight reparameterization module structure diagram is shown; Figure 5 A convolution feature fusion module structure diagram is shown; Figure 6 A block diagram of a helmet detection device based on improved YOLOv5 is shown. Figure 7 A structure diagram of a terminal is shown. DETAILED DESCRIPTION

[0026] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure can be embodied in various forms without being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0027] It should be understood that the sizes of the various portions shown in the drawings are not necessarily shown to scale.

[0028] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the application or its application or uses.

[0029] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein. However, where appropriate, such techniques, methods, and devices can be considered part of the specification.

[0030] It should be noted that like reference numerals and letters refer to like items in the following drawings, and thus, once an item is defined in one drawing, it is not necessary to discuss it further in subsequent drawings.

[0031] The embodiments of the present application can be applied to a computer system / server and can operate with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that can be suitable for use with computer system / server include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or the like.

[0032] The computer system / server can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and so on, that perform particular tasks or implement particular abstract data types. Computer system / server can operate in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in local or remote computer system storage media including memory storage devices.

[0033] The embodiment of the application provides a helmet detection method based on improved YOLOv5, as shown in the figure, which comprises the following steps: Figure 1 101. Collecting a real-time image of a target object as a target image based on an edge device.

[0034] Among them, the edge device can be a mobile law enforcement terminal, a traffic monitoring device, etc.; the target object is, for example, an electric vehicle driver, a motorcycle driver, etc. In the embodiment of the application, the current execution end can be a data processing module in the edge device such as a mobile law enforcement terminal or a traffic monitoring device.

[0035] It should be noted that the edge device can only collect a single static image; or collect a video stream and then extract a frame image from the video stream, which is not limited in the embodiment of the application.

[0036] 102. Performing helmet detection processing on the target object in the target image based on the helmet detection model trained by the edge device, to obtain the helmet detection result of the target object.

[0037] Among them, the helmet detection model is constructed by taking YOLOv5 neural network as the basic architecture, using lightweight reparameterization module (SimRepCSP) as the feature extraction layer, and using double convolution feature fusion module (C3_DualConv) as the feature fusion layer.

[0038] It should be noted that the lightweight reparameterization module has fewer model parameters and lower calculation amount, and using it as the feature extraction layer can effectively simplify the network structure of the YOLOv5 neural network, thereby improving the target detection rate; at the same time, using the double convolution feature fusion module improves the feature fusion ability of the model, so that the detection ability of the model on multiple scale targets is enhanced, thereby improving the detection precision and accuracy.

[0039] 103. Extracting the identification mark of the target object from the target image, mapping the identification mark with the helmet detection result, and sending the mapping result to a traffic monitoring database as a basis for traffic management data.​

[0040] The identification mark can be a license plate number or the like that can identify the identity of the target object. In the embodiments of the present application, the identification mark and the helmet detection result are sent to a traffic monitoring database in a mapping form, so that when the helmet detection result is that the helmet is not worn, the identity information of the target object is found and determined according to the identification mark, and the helmet detection result is sent to the target object's reserved mobile phone number or email in the form of a short message or an email, so as to notify the target object to handle the violation.

[0041] It should be noted that when the identification mark extraction fails, the facial feature image of the target object can also be extracted from the target image and compared with the facial feature image reserved in the identity information library to determine the identity information of the target object.

[0042] Compared with the prior art, the embodiments of the present application use a light-weight reparameterization module as a feature extraction layer, which simplifies the network structure of the traditional YOLOv5 neural network, so that the model parameters are fewer and the calculation amount is lower, thereby improving the target detection rate. At the same time, a double convolution feature fusion module is used as a feature fusion layer, which improves the poor information fusion ability of the traditional YOLOv5 neural network, enhances the detection ability of the model on multi-scale targets, and improves the detection precision and accuracy. The embodiments of the present application provide another helmet detection method based on improved YOLOv5, as shown in Figure 2 The method comprises the following steps. 201. Collecting a real-time image of a target object based on an edge device as a target image.

[0043] The embodiment step 201 can refer to the detailed description in the aforementioned embodiment step 101, which will not be repeated here.

[0044] 202. Performing size standardization processing and pixel normalization processing on the target image to obtain a target image that has completed preprocessing, so as to perform helmet detection processing on the target object in the target image that has completed preprocessing.

[0045] The size standardization processing is used to scale the target image to a specified fixed size to ensure that the input image and the model input layer have consistent dimensions. The pixel normalization processing is used to normalize the numerical range of image pixels from the conventional [0, 255] to the range of [0, 1] to improve the detection speed of the model.

[0046] 203. Performing resolution adjustment processing on the target image to obtain a target image that has completed resolution adjustment, so as to perform helmet detection processing on the target object in the target image that has completed resolution adjustment.

[0047] The resolution adjustment processing is used to reduce the resolution of the target image to reduce the amount of calculation, thereby improving the detection speed of the model.

[0048] It should be noted that the embodiment steps 202 and 203 can be executed separately or simultaneously, that is, 202 or 203 or 202 and 203 are executed.

[0049] 204, performing feature extraction processing by using a lightweight reparameterization module.

[0050] Correspondingly, the embodiment step 204 specifically includes: inputting the target image into the lightweight reparameterization module; using multiple convolution layers and multiple reparameterization convolution layers in the lightweight reparameterization module to perform multi-level feature transformation processing on the target image to obtain a multi-level feature transformation processing result; and performing fusion processing on the multi-level feature transformation processing result and the target image to obtain a multi-size feature map group, wherein the reparameterization convolution layer is reparameterized into a single convolution kernel in the inference process.

[0051] In the embodiment of the present application, in the lightweight reparameterization module, the target image is transmitted to two processing paths, i.e., a multi-level feature transformation processing path and a shortcut processing path; in the multi-level feature transformation processing path, the target image first enters the convolution layer for processing, which can be configured with a kernel size of 1 and a step size of 1, and then enters the reparameterization convolution layer for processing to obtain a multi-level feature transformation processing result. It should be noted that the reparameterization convolution layer has multiple branch structures during training, such as 3x3 convolution, 1x1 convolution, and identity mapping three branch structures, etc. In the inference process, the multiple branch weights can be equivalently fused into a single convolution kernel, such as a 3x3 convolution kernel, through structure reparameterization technology; in the shortcut processing path, the target image is not processed, and its output is still the target image; finally, the multi-level feature transformation processing result and the target image are fused and processed through the convolution layer and output to obtain a multi-size feature map group.

[0052] It should be noted that the lightweight reparameterization module realizes efficient feature extraction by constructing a main path containing a reparameterization convolution and a shortcut path, and finally performing splicing and fusion, which simplifies the network structure during inference through structure reparameterization technology while ensuring performance, and significantly reduces the calculation delay.

[0053] 205, performing feature fusion processing by using a double convolution feature fusion module.

[0054] Correspondingly, the embodiment step 205 specifically comprises: inputting the multi-size feature map group into the first branch and the second branch of the double convolution feature fusion module respectively; in the first branch, performing convolution transformation processing on the multi-size feature map group to obtain first branch output feature map; in the second branch, based on a plurality of bottleneck modules (BottleNeck), sequentially performing deep feature extraction operation on the multi-size feature map group, wherein in each bottleneck module, the multi-size feature map group is divided into a preset number of groups, at least one of the plurality of groups is subjected to parallel convolution operation to obtain parallel convolution result, the remaining groups are subjected to separate convolution operation to obtain separate convolution result, and the parallel convolution result and the separate convolution result are added to obtain second branch output feature map; performing channel dimension splicing processing and integration processing on the first branch output feature map and the second branch output feature map to obtain the multi-size feature map group after feature fusion.

[0055] In the embodiment of the application, the received multi-size feature map group (output result of embodiment step 204) is input into two branches respectively; the first branch is a shortcut branch, in which the multi-size feature map group is subjected to convolution transformation processing to obtain first branch output feature map; the second branch is a feature enhancement branch, in which the multi-size feature map group is first subjected to basic transformation by a convolution layer, and then input into a plurality of bottleneck modules stacked in series for deep feature extraction operation, and further, in the bottleneck module, the multi-size feature map group is divided into a preset number of groups, at least one of the plurality of groups is subjected to parallel convolution operation, i.e. 3x3 convolution and 1x1 convolution are performed simultaneously, to obtain parallel convolution result, while the remaining groups are subjected to separate convolution operation, i.e. only 1x1 convolution is performed, to obtain separate convolution result, and the parallel convolution result and the separate convolution result are added, i.e. element-wise addition, to obtain second branch output feature map; finally, the first branch output feature map and the second branch output feature map are subjected to channel dimension splicing processing and integration processing to obtain the multi-size feature map group after feature fusion with increased number of channels.

[0056] It should be noted that the double convolution feature fusion module effectively realizes the reservation and enhancement of multi-scale features by constructing a shortcut branch and a feature enhancement branch containing a double convolution bottleneck module, and performing splicing and fusion at the end, improves the feature fusion capability of the model neck network, and at the same time, takes into account the calculation efficiency.

[0057] 206, based on the multi-size feature map group after feature fusion, performing position positioning and category prediction on the target object to obtain a helmet detection result of the target object.

[0058] In the embodiment of the application, using the feature map after feature enhancement for position positioning and category prediction can effectively improve the accuracy of detection.

[0059] In one embodiment of the present application, in order to further limit and illustrate, as shown in Figure 3 Before the embodiment step 102 performs the helmet detection processing on the target object in the target image based on the helmet detection model deployed in the edge device and completed model training, the embodiment method further comprises: 301. Take YOLOv5 neural network as the basic architecture, use lightweight reparameterization module as the feature extraction layer, and use double convolution feature fusion module as the feature fusion layer to construct the initial helmet detection model.

[0060] As shown in Figure 4 from left to right in the figure are convolution layer 1, convolution layer 2, reparameterization convolution layer, fusion layer and convolution layer 3. It can be understood that convolution layer 2 and reparameterization convolution layer form a multi-level feature transformation processing path, and the path between convolution layer 2 and fusion layer is a shortcut processing path; the structure of the convolution feature fusion module is as shown in Figure 5 The left branch in the figure is the first branch, which only contains a convolution layer. The right branch is the second branch, which, from top to bottom, is a convolution layer, a bottleneck module, a fusion layer and a convolution layer.

[0061] 302. Obtain a training sample set.

[0062] Among them, the sample image is labeled with the position and category of the detection object, and the category includes wearing a helmet and not wearing a helmet.

[0063] 303. Based on the initial helmet detection model, perform helmet detection processing on the detection object in each sample image to obtain multiple initial helmet detection results.

[0064] 304. Calculate the loss function value between the multiple initial helmet detection results and the multiple sample image annotation results.

[0065] Among them, the loss function value is composed of an improved bounding box regression loss value, a confidence loss value and a classification loss value. The improved bounding box regression loss value is calculated based on the following formula,

[0066] represents the improved bounding box regression loss value, L MPDIoU represents the loss value of MPDIoU, IoU represents the intersection over union between the predicted box and the real box, IoU Inner represents the intersection over union between the Inner predicted box and the Inner real box.

[0067] It should be noted that in existing technologies, the CIoU bounding box regression loss function is typically used to calculate the bounding box regression loss value. However, since its aspect ratio is defined as a relative value rather than an absolute value, when the aspect ratio of the predicted box (i.e., Pred) is the same as that of the ground truth box (i.e., GT), the actual width and height values ​​may differ. CIoU cannot distinguish this difference in such cases, leading to compromised regression accuracy and resulting in missed detections and false positives. In the embodiments of this application, an improved bounding box regression loss function can solve the above problems, thereby improving the model convergence performance.

[0068] 305. Based on the loss function value, backpropagate and iteratively update the model parameters of the initial helmet detection model until the preset iteration stopping condition is reached, and obtain the helmet detection model that has completed model training.

[0069] The preset iteration stopping condition can be either the convergence of the loss function value or the reaching of a preset number of iterations.

[0070] 306. Deploy the helmet detection model that has completed model training on an edge device to perform helmet detection on the edge device.

[0071] Accordingly, step 306 of the embodiment specifically includes: converting the helmet detection model that has completed model training into an intermediate format helmet detection model; using a model optimizer to perform model optimization and serialization processing on the intermediate format helmet detection model to obtain the engine file corresponding to the helmet detection model that has completed model training, wherein the model optimization processing includes layer fusion optimization processing, kernel tuning processing and computational precision adjustment processing; and deploying the engine file in an edge device.

[0072] The intermediate format can be ONNX format, used to record the operator execution order, tensor dimensions, and computational dependencies, thereby solidifying the computational process and avoiding the computational power consumed by rebuilding the computation graph each time inference, thus improving the target detection rate. Layer fusion optimization processing is used to fuse the model layer sequences that allow fusion into a single kernel function to simplify the number of kernel function calls, thereby reducing memory read / write and kernel scheduling overhead and freeing up computational power. Kernel tuning processing is used to select the engine kernel function with the fastest execution speed on the graph processing module of the current edge device from multiple pre-generated engine kernel functions to improve the target detection rate. Computational precision adjustment processing is used to perform half-precision calculations on non-critical paths while ensuring detection accuracy, thereby reducing the model size. The engine file is used to record the serialized model running scheme, so that during deployment, only the engine file needs to be deserialized and loaded to directly perform helmet detection processing, without having to perform the time-consuming model loading and optimization process again, further ensuring the rapid start-up and efficient operation of the model.

[0073] The application provides a helmet detection method based on improved YOLOv5. First, real-time images of a target object are collected based on an edge device as target images. Second, a helmet detection model trained in the edge device is used to perform helmet detection processing on the target object in the target image to obtain a helmet detection result of the target object. The helmet detection model is based on a YOLOv5 neural network as a basic architecture, uses a lightweight reparameterization module (SimRepCSP) as a feature extraction layer, and uses a dual convolution feature fusion module (C3_DualConv) as a feature fusion layer. Finally, an identification of the target object is extracted from the target image, the identification is mapped with the helmet detection result, and the mapping result is sent to a traffic monitoring database. Compared with the prior art, the lightweight reparameterization module is used as the feature extraction layer to simplify the network structure of the traditional YOLOv5 neural network, so that the model parameters are fewer and the calculation amount is lower, thereby improving the target detection rate. Meanwhile, the dual convolution feature fusion module is used as the feature fusion layer to improve the poor information fusion capability of the traditional YOLOv5 neural network, enhance the detection capability of the model on multi-scale targets, and improve the detection precision and accuracy. Further, as an implementation of the method shown in the above Figure 1 The application embodiment provides a helmet detection device based on improved YOLOv5, as shown in the above Figure 6 The device comprises: an image collection module 41, a helmet detection module 42, and a detection result sending module 43. The image collection module 41 is configured to collect real-time images of a target object based on an edge device as target images. The helmet detection module 42 is configured to perform helmet detection processing on the target object in the target image based on a helmet detection model trained in the edge device to obtain a helmet detection result of the target object. The helmet detection model is based on a YOLOv5 neural network as a basic architecture, uses a lightweight reparameterization module (SimRepCSP) as a feature extraction layer, and uses a dual convolution feature fusion module (C3_DualConv) as a feature fusion layer. The detection result sending module 43 is configured to extract an identification of the target object from the target image, map the identification with the helmet detection result, and send the mapping result to a traffic monitoring database.

[0074] In a specific application scenario, the helmet detection module is configured to: input the target image into the lightweight reparameterization module; The target image is subjected to multi-level feature transformation processing by using the plurality of convolution layers and the plurality of reparameterization convolution layers in the lightweight reparameterization module, to obtain a multi-level feature transformation processing result. The multi-level feature transformation processing result and the target image are subjected to fusion processing to obtain a multi-size feature map group, wherein the reparameterization convolution layer is reparameterized into a single convolution kernel in an inference process.

[0075] In a specific application scenario, the helmet detection module is further configured to: The multi-size feature map group is input into a first branch and a second branch of the double-convolution feature fusion module respectively. In the first branch, the multi-size feature map group is subjected to convolution transformation processing to obtain a first-branch output feature map. In the second branch, a plurality of BottleNeck modules are used to sequentially perform deep feature extraction operations on the multi-size feature map group, wherein in each BottleNeck module, the multi-size feature map group is divided into a preset number of groups, at least one group of the plurality of groups is subjected to parallel convolution operation to obtain a parallel convolution result, the remaining groups are subjected to separate convolution operation to obtain separate convolution results, and the parallel convolution result and the separate convolution results are added to obtain a second-branch output feature map. The first-branch output feature map and the second-branch output feature map are subjected to channel dimension splicing processing and integration processing to obtain a multi-size feature map group after feature fusion, and the target object is subjected to position positioning and class prediction based on the multi-size feature map group after feature fusion.

[0076] In a specific application scenario, the device further comprises a model training module before the helmet detection module, which comprises: A model training unit is configured to use a YOLOv5 neural network as a basic architecture, use a lightweight reparameterization module as a feature extraction layer, and use a double-convolution feature fusion module as a feature fusion layer to construct an initial helmet detection model, obtain a training sample set, wherein the sample images are labeled with the positions and classes of the detection objects, and the classes include wearing a helmet and not wearing a helmet, perform helmet detection processing on the detection objects in each sample image based on the initial helmet detection model to obtain a plurality of initial helmet detection results, and calculate a loss function value between the plurality of initial helmet detection results and a plurality of sample image labeling results, wherein the loss function value is composed of an improved bounding box regression loss value, a confidence loss value, and a classification loss value, and the improved bounding box regression loss value is calculated based on the following formula,

[0077] denotes an improved bounding box regression loss value, L MPDIoU denotes a loss value of MPDIoU, IoU denotes an intersection over union between a predicted box and a real box, IoU Inner denotes an intersection over union between an Inner predicted box and an Inner real box; based on the loss function value, model parameters of the initial helmet detection model are back propagated and iteratively updated until a preset iteration stopping condition is reached, obtaining a helmet detection model that has completed model training; a deployment unit, configured to deploy the helmet detection model that has completed model training in the edge device, to perform helmet detection in the edge device.

[0078] In a specific application scenario, the deployment unit is configured to: convert the helmet detection model that has completed model training into an intermediate format helmet detection model; perform model optimization processing and serialization processing on the intermediate format helmet detection model using a model optimizer, to obtain an engine file corresponding to the helmet detection model that has completed model training, wherein the model optimization processing includes layer fusion optimization processing, kernel tuning processing, and calculation precision adjustment processing; deploy the engine file in the edge device.

[0079] In a specific application scenario, before the helmet detection module, the apparatus further includes an image preprocessing module configured to: perform size standardization processing and pixel normalization processing on the target image, to obtain a preprocessed target image, to perform helmet detection processing on a target object in the preprocessed target image.

[0080] In a specific application scenario, before the helmet detection module, the apparatus further includes an image resolution adjustment module configured to: perform resolution adjustment processing on the target image, to obtain a target image that has completed resolution adjustment, to perform helmet detection processing on a target object in the target image that has completed resolution adjustment.

[0081] The application provides a helmet detection device based on improved YOLOv5. First, real-time images of a target object are collected based on an edge device as target images. Second, a helmet detection model trained by the edge device is used to detect the target object in the target image to obtain a helmet detection result of the target object. The helmet detection model is based on a YOLOv5 neural network, uses a lightweight reparameterization module (SimRepCSP) as a feature extraction layer, and uses a dual convolution feature fusion module (C3_DualConv) as a feature fusion layer. Finally, an identification of the target object is extracted from the target image, the identification is mapped with the helmet detection result, and the mapping result is sent to a traffic monitoring database. Compared with the prior art, the lightweight reparameterization module is used as the feature extraction layer to simplify the network structure of the traditional YOLOv5 neural network, so that the model parameters are fewer and the calculation amount is lower, thereby improving the target detection rate. Meanwhile, the dual convolution feature fusion module is used as the feature fusion layer to improve the poor information fusion capability of the traditional YOLOv5 neural network, enhance the detection capability of the model on multi-scale targets, and improve the detection precision and accuracy. According to an embodiment of the application, a storage medium is provided, which stores at least one executable instruction. The computer executable instruction can execute the helmet detection method based on improved YOLOv5 in any method embodiment described above.

[0082] Based on such understanding, the technical solution of the application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in various implementation scenarios of the application.

[0083] Figure 7 A structural schematic diagram of a terminal according to an embodiment of the application is shown, and the specific implementation of the terminal is not limited in the specific embodiments of the application.

[0084] As shown in Figure 7 , the terminal can include a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0085] The processor 502, the communications interface 504, and the memory 506 can communicate with each other through the communications bus 508.

[0086] The communication interface 504 is configured to communicate with network elements such as clients or other servers.

[0087] The processor 502 is configured to execute the program 510, and specifically can execute the related steps in the above-described helmet detection method based on the improved YOLOv5.

[0088] Specifically, the program 510 can include program code including computer operation instructions.

[0089] The processor 502 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computer device can be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0090] The memory 506 is configured to store the program 510. The memory 506 can include a high-speed RAM memory, and can also include a non-volatile memory such as at least one disk memory.

[0091] The program 510 can specifically be configured to cause the processor 502 to perform the following operations: Collect a real-time image of a target object based on an edge device, as a target image; Perform helmet detection processing on the target object in the target image based on a helmet detection model trained by the edge device, to obtain a helmet detection result of the target object, wherein the helmet detection model is constructed based on a YOLOv5 neural network as a basic architecture, a light-weight reparameterization module as a feature extraction layer, and a double convolution feature fusion module as a feature fusion layer; Extract an identification of the target object from the target image, map the identification and the helmet detection result, and send a mapping result to a traffic monitoring database.

[0092] The storage medium can also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the entity device of the above-described helmet detection method based on the improved YOLOv5, and supports the running of the information processing program and other software and / or programs. The network communication module is configured to realize communication between the components in the storage medium, and communication with other hardware and software in the information processing entity device.

[0093] The various embodiments described in this specification are intended to be illustrative only and in no way limit the scope of the application. One skilled in the art will readily recognize from the disclosure herein, that alternative embodiments of the methods and systems of the present application can be constructed from a wide variety of different components and configurations of components. The methods and systems of the present application are therefore not limited to the specific embodiments described herein, but rather, the scope of the present application is to be determined entirely by the following claims, which are to be interpreted in accordance with the principles of patent law.

[0094] The methods and systems of the present application can be implemented in a number of ways. For example, the methods and systems of the present application can be implemented using software, hardware, firmware, or any combination of these methods. Any features described as modules, components or steps can be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. Any features described as a process or method can be implemented as a whole in one device or circuit, or as parts in two or more devices or circuits. The order of any steps described is not limited to the order described, unless otherwise specifically stated.

[0095] It should be apparent to those skilled in the art that the modules or steps of the application described above can be implemented with a general computing device, which can be centralized on a single computing device or distributed across multiple computing devices, and optionally implemented with program code executable by a computing device, which can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be performed in an order different from that shown or described, or made into individual integrated circuit modules, or made into a single integrated circuit module, or a plurality of modules or steps. Thus, the present application is not limited to any particular hardware and software combination.

[0096] The preferred embodiments of the present application described above are intended to be illustrative only and not limiting. Numerous modifications and adaptations thereof will be apparent to those skilled in the art without departing from the spirit and scope of the present application. All such modifications and adaptations are intended to be within the scope of the present application.

Claims

1. A helmet detection method based on an improved YOLOv5, characterized in that, include: Real-time images of the target object are acquired using edge devices and used as the target image. Based on the helmet detection model that has been trained and deployed in the edge device, helmet detection processing is performed on the target object in the target image to obtain the helmet detection result of the target object. The helmet detection model is built with YOLOv5 neural network as the basic architecture, and uses a lightweight reparameterization module (SimRepCSP) as the feature extraction layer and a dual convolution feature fusion module (C3_DualConv) as the feature fusion layer. The identification identifier of the target object is extracted from the target image, the identification identifier is mapped to the helmet detection result, and the mapping result is sent to the traffic monitoring database.

2. The method according to claim 1, characterized in that, Feature extraction is performed using a lightweight, heavily parameterized module, including: The target image is input into the lightweight reparameterization module; The target image is subjected to multi-level feature transformation processing by using multiple convolutional layers and multiple reparameterized convolutional layers in the lightweight reparameterization module, and the multi-level feature transformation processing result is obtained. The multi-level feature transformation processing result is fused with the target image to obtain a multi-size feature map group, wherein the reparameterized convolutional layer is reparameterized into a single convolutional kernel during inference.

3. The method according to claim 1, characterized in that, Feature fusion processing is performed using a dual-convolutional feature fusion module, including: The multi-size feature map groups are respectively input into the first branch and the second branch of the dual convolutional feature fusion module; In the first branch, the multi-size feature map group is subjected to convolution transformation to obtain the output feature map of the first branch; In the second branch, based on multiple bottleneck modules, deep feature extraction operations are sequentially performed on the multi-size feature map group. In each bottleneck module, the multi-size feature map group is divided into a preset number of groups, and at least one of the groups is subjected to parallel convolution operation to obtain a parallel convolution result. The remaining groups are subjected to individual convolution operation to obtain individual convolution results. The parallel convolution result and the individual convolution result are added together to obtain the output feature map of the second branch. The first branch output feature map and the second branch output feature map are spliced ​​and integrated along the channel dimension to obtain a multi-size feature map group after feature fusion. The target object is located and its category is predicted based on the multi-size feature map group after feature fusion.

4. The method according to claim 1, characterized in that, Before performing helmet detection processing on the target object in the target image based on the helmet detection model deployed in the edge device that has completed model training, and obtaining the helmet detection result of the target object, the method further includes: Using YOLOv5 neural network as the basic architecture, and utilizing a lightweight reparameterization module as the feature extraction layer and a dual convolution feature fusion module as the feature fusion layer, an initial helmet detection model was constructed. Obtain a training sample set, in which the location and category of the detection object are marked in the sample images, and the categories include wearing a helmet and not wearing a helmet; Based on the initial helmet detection model, helmet detection processing is performed on the detection objects in each of the sample images to obtain multiple initial helmet detection results; The loss function value between the multiple initial helmet detection results and the multiple sample image annotation results is calculated. The loss function value consists of the improved bounding box regression loss value, the confidence loss value, and the classification loss value. The improved bounding box regression loss value is calculated based on the following formula. L represents the improved bounding box regression loss value. MPDIoU MPDIoU represents the loss value, and IoU represents the intersection-over-union ratio between the predicted and ground truth boxes. Inner This represents the intersection-over-union ratio (IoU) between the predicted inner bounding box and the actual inner bounding box. Based on the loss function value, backpropagation is performed and the model parameters of the initial helmet detection model are iteratively updated until the preset iteration stopping condition is met, thus obtaining a helmet detection model that has completed model training. The helmet detection model that has completed model training is deployed in the edge device to perform helmet detection in the edge device.

5. The method according to claim 4, characterized in that, Deploying the trained helmet detection model in the edge device includes: The helmet detection model that has completed model training is converted into an intermediate format helmet detection model; The intermediate format helmet detection model is optimized and serialized using a model optimizer to obtain the engine file corresponding to the helmet detection model that has completed model training. The model optimization process includes layer fusion optimization, kernel tuning, and computational precision adjustment. The engine file is deployed on the edge device.

6. The method according to claim 1, characterized in that, Before performing helmet detection processing on the target object in the target image based on the helmet detection model deployed in the edge device that has completed model training, and obtaining the helmet detection result of the target object, the method further includes: The target image is subjected to size normalization and pixel normalization to obtain a preprocessed target image, and helmet detection processing is performed on the target object in the preprocessed target image.

7. The method according to claim 1, characterized in that, Before performing helmet detection processing on the target object in the target image based on the helmet detection model deployed in the edge device that has completed model training, and obtaining the helmet detection result of the target object, the method further includes: The target image is subjected to resolution adjustment processing to obtain a target image with adjusted resolution, and helmet detection processing is performed on the target object in the target image with adjusted resolution.

8. A helmet detection device based on an improved YOLOv5, characterized in that, include: The image acquisition module is used to acquire real-time images of the target object based on the edge device, and use them as the target image; The helmet detection module is used to perform helmet detection processing on the target object in the target image based on the helmet detection model that has been trained and deployed in the edge device, and to obtain the helmet detection result of the target object. The helmet detection model is built with YOLOv5 neural network as the basic architecture, and uses a lightweight reparameterization module (SimRepCSP) as the feature extraction layer and a dual convolution feature fusion module (C3_DualConv) as the feature fusion layer. The detection result sending module is used to extract the identification mark of the target object from the target image, map the identification mark with the helmet detection result, and send the mapping result to the traffic monitoring database.

9. A storage medium storing at least one executable instruction, characterized in that, The executable instructions cause the processor to perform the operations corresponding to the helmet detection method based on the improved YOLOv5 as described in any one of claims 1-7.

10. A terminal, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, characterized in that the executable instruction causes the processor to perform the operation corresponding to the helmet detection method based on the improved YOLOv5 as described in any one of claims 1-7.

Images