Safety helmet wearing detection method and device, electronic device, and computer-readable storage medium

Abstract

The present application relates to a helmet wearing detection method and device, an electronic device, and a computer-readable storage medium, including: taking the acquired current image frame and the first preset number of image frames before the current image frame as a circular queue. Performing face detection on the image frames in the circular queue to obtain the face detection result of each image frame in the circular queue, and inputting the face detection result of each image frame in the circular queue into a preset classifier model to calculate the helmet wearing probability of each image frame. Correcting the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue. In this way, the accuracy of the final helmet wearing probability of the current image frame obtained is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for detecting the wearing of a helmet, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the continuous deepening of urbanization, people's pace of life is getting faster and faster. The rapid development of the Internet has activated the takeaway and express delivery markets, and people enjoy the convenience brought to their lives by online shopping and food delivery. However, while being convenient, since the takeaway and express delivery markets have strict time limits for delivery and dispatch services, delivery personnel often have irregular driving behaviors such as not wearing helmets and making phone calls during driving, which greatly increases the severity and probability of traffic accidents. Studies have shown that drivers wearing helmets will greatly reduce the incidence of traffic accidents. Therefore, it is necessary to check and supervise whether drivers are wearing helmets. The traditional helmet wearing detection method usually uses cameras deployed at the entrances and exits of public roads and major pedestrian and traffic channels in the city to conduct real-time statistics on the status of two-wheeled or three-wheeled motor vehicle drivers wearing helmets. However, the accuracy of this detection method is low. Summary of the invention

[0003] The embodiments of the present application provide a method, device, electronic device, and computer-readable storage medium for detecting the wearing of a helmet, which can improve the accuracy of detecting the wearing of a helmet.

[0004] A method for detecting the wearing of a safety helmet, comprising:

[0005] Using the acquired current image frame and a first preset number of image frames before the current image frame as a circular queue;

[0006] Performing face detection on the image frames in the circular queue to obtain a face detection result for each image frame in the circular queue;

[0007] Inputting the face detection result of each image frame in the circular queue into a preset classifier model to obtain the helmet wearing probability of each image frame;

[0008] The helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0009] In one embodiment, the step of correcting the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue includes:

[0010] According to the helmet wearing probability of each image frame in the circular queue and the corresponding weights, the helmet wearing probability of the current image frame in the circular queue is corrected to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0011] In one embodiment, the step of correcting the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue and the corresponding weights thereof to obtain the corrected helmet wearing probability of the current image frame in the circular queue comprises:

[0012] According to the shooting frame rate of the image frames in the circular queue and the frame number interval between each image frame in the circular queue and the current image frame, a weight is configured for each image frame in the circular queue;

[0013] The weighted sum of the helmet wearing probability of each image frame in the circular queue and the corresponding weight is used as the corrected helmet wearing probability of the current image frame in the circular queue.

[0014] In one embodiment, performing face detection on the image frames in the circular queue to obtain a face detection result for each image frame in the circular queue includes:

[0015] Performing face detection on the image frames in the circular queue to obtain a face region of each image frame in the circular queue;

[0016] The face area is enlarged by a preset ratio to obtain a target area, and the target area includes an area where a helmet is worn.

[0017] In one embodiment, the generation process of the preset classifier model includes:

[0018] Inputting a training set into an initialized lightweight deep neural network model for training, wherein the training set includes less than a second preset number of image frames of images of people wearing and not wearing a helmet;

[0019] The parameters of the initialized lightweight deep neural network model are adjusted through the loss function to obtain the preset classifier model.

[0020] In one embodiment, the method further comprises:

[0021] When it is determined that the corrected helmet wearing probability of the current image frame in the circular queue is greater than the first threshold, it is detected that the person in the current image frame in the circular queue is wearing a helmet.

[0022] In one embodiment, the method further comprises:

[0023] Determine whether the latest helmet wearing probability of each image frame in the circular queue is greater than a first threshold, and obtain a determination result;

[0024] According to the judgment result, the number of the latest image frames in the circular queue whose probability of wearing a helmet is greater than a first threshold is obtained;

[0025] When the number of the image frames is greater than a preset number threshold, it is detected that the person in the current image frame in the circular queue is wearing a safety helmet.

[0026] In one of the embodiments, after obtaining the corrected helmet wearing probability of the current image frame in the circular queue, the method further includes:

[0027] Acquire the next image frame in the circular queue as the current image frame, and delete the earliest image frame in the circular queue to update the circular queue;

[0028] Obtaining the helmet wearing probability of the current image frame in the circular queue;

[0029] The helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0030] A safety helmet wearing detection device, comprising:

[0031] A circular queue acquisition module, used for acquiring a current image frame and a first preset number of image frames before the current image frame as a circular queue;

[0032] A face detection module, used to perform face detection on the image frames in the circular queue to obtain a face detection result for each image frame in the circular queue;

[0033] A helmet wearing probability calculation module, used for inputting the face detection result of each image frame in the circular queue into a preset classifier model to calculate the helmet wearing probability of each image frame;

[0034] The helmet wearing probability correction module is used to correct the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0035] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0036] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0037] The above-mentioned helmet wearing detection method, device, electronic device, and computer-readable storage medium use the acquired current image frame and the first preset number of image frames before the current image frame as a circular queue. Perform face detection on the image frames in the circular queue to obtain the face detection result of each image frame in the circular queue, and input the face detection result of each image frame in the circular queue into a preset classifier model to calculate the helmet wearing probability of each image frame. Correct the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0038] Because the change from the state of wearing a helmet to the state of not wearing a helmet or vice versa needs to be completed within a certain period of time, when calculating the final helmet wearing probability of each current image frame, the possible influence of the helmet wearing probability of the first preset number of image frames before the current image frame on the final helmet wearing probability of the current image frame is considered accordingly. In this way, the accuracy of the final helmet wearing probability of the current image frame is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A diagram showing an application environment of a helmet wearing detection method in one embodiment;

[0041] Figure 2 A flowchart of a helmet wearing detection method in one embodiment;

[0042] Figure 3 A flowchart of a method for correcting the helmet wearing probability of a current image frame in one embodiment;

[0043] Figure 4A A schematic diagram of performing face detection on an image frame to obtain a rectangular frame in one embodiment;

[0044] Figure 4B It is a schematic diagram of performing face detection on an image frame to obtain a rectangular frame in another embodiment;

[0045] Figure 5 A flowchart of a method for detecting helmet wearing in another embodiment;

[0046] Figure 6 A flowchart of a method for detecting helmet wearing in yet another embodiment;

[0047] Figure 7 A flowchart of a helmet wearing detection method in a specific embodiment;

[0048] Figure 8 is a structural block diagram of a safety helmet wearing detection device in one embodiment;

[0049] Fig. 9 is a structural block diagram of a safety helmet wearing detection device in another embodiment;

[0050] Fig.10 FIG. 1 is a schematic diagram of the internal structure of an electronic device in an embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] It can be understood that the terms "first", "second", etc. used in the present application can be used in this article to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element.

[0053] Figure 1 FIG. 1 is a schematic diagram of an application environment of a helmet wearing detection method in a charging process according to an embodiment. Figure 1As shown, the application environment includes an electronic device 120 and a server 140. The electronic device 120 has a camera function, and can analyze and process the photos or videos taken, determine whether the people in the photos or videos taken are wearing helmets, and upload the images of people who are not wearing helmets to the database in the server 140 for storage, so as to record the people who are not wearing helmets, and warn and punish them. The electronic device 120 can be installed on a vehicle (two-wheeled, three-wheeled vehicles and cars) for delivery of express or take-out, so as to take photos or videos of the driver or accompanying personnel. Of course, the electronic device 120 can also be installed in scenes such as construction sites where it is necessary to detect whether people are wearing helmets, and take photos or videos of the construction sites. The electronic device 120 can use the current image frame obtained and the first preset number of image frames before the current image frame as a circular queue. Perform face detection on the image frames in the circular queue to obtain the face detection results of each image frame in the circular queue, and input the face detection results of each image frame in the circular queue into a preset classifier model to calculate the probability of wearing a helmet for each image frame. The helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue. It is understandable that the electronic device 120 may be not limited to various mobile phones, computers, portable devices, digital cameras and other devices with photo taking and processing functions, but may also be an embedded device with a camera.

[0054] Figure 2 FIG. 1 is a flow chart of a method for detecting helmet wearing in one embodiment. Figure 2 As shown, the helmet wearing detection method includes steps 220 to 280, and is applied to an electronic device or a server.

[0055] Step 220: The acquired current image frame and a first preset number of image frames before the current image frame are used as a circular queue.

[0056] When the electronic device continuously shoots a person who needs to be judged whether he is wearing a helmet, the current image frame and the first preset number of image frames before the current image frame are obtained, and the obtained image frames are used as a circular queue. That is, the circular queue includes the current image frame and the first preset number of image frames before the current image frame. Here, the first preset number can be determined based on the quality of the image frame actually shot, whether the person in the image frame is in motion, etc. The first preset number can be selected from n image frames before the current image frame. For example, the first preset number can be selected from 4 image frames before the current image frame. Of course, this application is not limited to this.

[0057] The circular queue is updated synchronously with the continuous updating of the current image frame. For example, the first preset number can select 4 image frames before the current image frame. The circular queue after each update includes the current image frame pre0, the previous frame pre1 of the current image frame, the previous two frames pre2 of the current image frame, the previous three frames pre3 of the current image frame, and the previous four frames pre4 of the current image frame. However, as time goes by, the current image frame constantly becomes the previous frame of the current image frame.

[0058] Step 240, performing face detection on the image frames in the circular queue to obtain a face detection result for each image frame in the circular queue.

[0059] The face detection model is used to perform face detection on the image frames in the circular queue to obtain the face position information of each image frame in the circular queue. For example, the face position information can be the position information of the four pixels located farthest up, down, left and right on the face area. Connecting these four pixels forms a rectangular frame, and the rectangular frame is expanded in the direction of the area where the helmet is worn, so that the expanded rectangular frame includes the face area and the area where the helmet is worn, and the expanded rectangular frame is the face detection result. The face detection model is becoming more and more mature, so the detection speed and detection accuracy of the currently mature face recognition model to detect the area including the face area and the area where the helmet is worn are significantly better than the algorithm that directly detects the helmet area. Therefore, the present application adopts the face detection model to detect the face, which can effectively improve the detection speed and accuracy of the helmet wearing area.

[0060] When the rectangular frame is expanded in the direction of the area where the helmet is worn, it is necessary to refer to the ratio of the face area to the helmet area in the image frame and the shooting angle of the image frame and other factors to expand. For example, when shooting directly, the ratio of the face area to the helmet area is about 2:1. At this time, after obtaining the rectangular frame of the face area, the rectangular frame can be expanded upward by 1 / 2. When shooting from an upward angle (the camera is located in the lower front of the human body), it is assumed that the ratio of the face area to the helmet area is about 5:1. Then, after obtaining the rectangular frame of the face area, the rectangular frame can be expanded upward by 1 / 5. Through this method, the rectangular frame corresponding to the face area is first detected, and then the rectangular frame is expanded to obtain a rectangular frame including the face area and the helmet area that may be included. Because the accuracy of the rectangular frame of the face area obtained by face detection is high, the accuracy of the detection frame including the face area and the helmet area that may be included after the expansion is also high.

[0061] Step 260: Input the face detection result of each image frame in the circular queue into a preset classifier model to calculate the helmet wearing probability of each image frame.

[0062] The enlarged rectangular frame of each image frame in the circular queue is obtained, and the enlarged rectangular frame of each image frame in the circular queue is input into a preset classifier model to calculate the helmet wearing probability of each image frame. The preset classifier model is a classifier model obtained by inputting a preset training set into a neural network model for training.

[0063] For example, the first preset number can select 4 image frames before the current image frame, and then obtain the helmet wearing probability P(pre0) corresponding to the current image frame pre0 in the circular queue, the helmet wearing probability P(pre1) corresponding to the previous frame pre1 of the current image frame, the helmet wearing probability P(pre2) corresponding to the previous two frames pre2 of the current image frame, the helmet wearing probability P(pre3) corresponding to the previous three frames pre3 of the current image frame, and the helmet wearing probability P(pre4) corresponding to the previous four frames pre4 of the current image frame.

[0064] Step 280, correcting the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue, to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0065] Because the change from the state of wearing a helmet to the state of not wearing a helmet or vice versa needs to be completed within a certain period of time, it is a gradual process. Obviously, the helmet wearing probability of the image frame that is closer to the current image frame in time has a greater impact on the helmet wearing probability of the current image frame. That is, as long as the frame number interval between the previous image frame and the current image frame is less than the preset threshold, if the previous image frame is wearing a helmet, then the probability of wearing a helmet in the current image frame is very high. Therefore, the helmet wearing probability of the current image frame in the circular queue can be corrected according to the helmet wearing probability of each image frame in the circular queue, and the corrected helmet wearing probability of the current image frame in the circular queue is obtained.

[0066] In the embodiments of the present application, the detection speed and accuracy of the area including the face area and the area wearing the helmet are detected by the currently mature face recognition model, which are obviously better than the algorithm for directly detecting the helmet area. Therefore, the face detection model used in the present application to detect the face can effectively improve the detection speed and accuracy of the helmet wearing area. In addition, because the change from the state of wearing a helmet to the state of not wearing a helmet or vice versa requires a certain period of time to complete, it is a gradual process. Therefore, when calculating the final helmet wearing probability of each current image frame, the possible influence of the helmet wearing probability of the first preset number of image frames before the current image frame on the final helmet wearing probability of the current image frame is considered accordingly. In this way, the accuracy of the final helmet wearing probability of the current image frame is improved.

[0067] In one embodiment, the helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue, including:

[0068] According to the helmet wearing probability of each image frame in the circular queue and their corresponding weights, the helmet wearing probability of the current image frame in the circular queue is corrected to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0069] Specifically, it is obvious that the helmet wearing probability of an image frame that is closer to the current image frame in time has a greater impact on the helmet wearing probability of the current image frame. Therefore, a higher weight is assigned to an image frame that is closer to the current image frame in the circular queue. When correcting the helmet wearing probability of the current image frame calculated by a preset classifier, the helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue and their corresponding weights. This allows the helmet wearing probability of the current image frame to be more affected by the helmet wearing probability of the image frame that is closer to the current image frame. Thus, the accuracy of the corrected helmet wearing probability of the current image frame is improved.

[0070] In the embodiment of the present application, it is obvious that the helmet wearing probability of an image frame that is closer to the current image frame in time has a greater impact on the helmet wearing probability of the current image frame. Therefore, a higher weight is assigned to an image frame that is closer to the current image frame in the circular queue. Then, according to the helmet wearing probability of each image frame in the circular queue and the corresponding weight, the helmet wearing probability of the current image frame in the circular queue is corrected. Thus, the accuracy of the corrected helmet wearing probability of the current image frame is improved.

[0071] In one embodiment, Figure 3 As shown, according to the helmet wearing probability of each image frame in the circular queue and the corresponding weights, the helmet wearing probability of the current image frame in the circular queue is corrected to obtain the corrected helmet wearing probability of the current image frame in the circular queue, including:

[0072] Step 320, configuring a weight for each image frame in the circular queue according to the shooting frame rate of the image frames in the circular queue and the frame number interval between each image frame in the circular queue and the current image frame;

[0073] Step 340 : The weighted sum of the helmet wearing probability of each image frame in the circular queue and the corresponding weight is used as the corrected helmet wearing probability of the current image frame in the circular queue.

[0074] The frame rate of the image frame is the number of pictures taken by the camera per second. These pictures are played continuously to form a dynamic video. Frame rate = frames / time, the unit is frames per second (f / s, frames per second, fps). Frame rate is a measure used to measure the number of displayed frames. Generally, fps is used to describe how many frames a video, electronic drawing or game plays per second.

[0075] Specifically, a weight is configured for each image frame in the circular queue according to the shooting frame rate of the image frames in the circular queue and the frame interval between each image frame in the circular queue and the current image frame. Assume that the maximum shooting frame rate of the camera is maxfps, and the shooting frame rate of the camera when shooting the image frames in the circular queue is fps. Then the weight of each image frame in the circular queue is configured by the following formula:

[0076]

[0077]

[0078]

[0079] in, is the weight of each image frame in the circular queue, p(cur) refers to the probability of wearing a helmet after correction of the current image frame pre0, and i refers to the frame interval between the image frame and the current image frame. For example, assuming that the circular queue includes 5 frames, then i = 0, that is, the probability of wearing a helmet corresponding to the current image frame pre0 P(pre0), the weight is Then i=1, that is, the probability of wearing a helmet corresponding to the previous frame pre1 of the current image frame, P(pre1), and the weight is Then i=2, that is, the probability of wearing a helmet corresponding to the first two frames pre2 of the current image frame, P(pre2), and the weight is Then i=3, that is, the probability of wearing a helmet corresponding to the first three frames pre3 of the current image frame, P(pre3), and the weight is Then i=4, that is, the probability of wearing a helmet corresponding to the first four frames pre4 of the current image frame, P(pre4), and the weight is

[0080] Obviously, The maximum weight is assigned to the helmet wearing probability P(pre0) corresponding to the current image frame pre0 through the index ni of A, and the minimum weight is assigned to the helmet wearing probability P(pre4) corresponding to pre4, which is farthest from the current image frame. ni represents the frame interval between each image frame in the circular queue and the current image frame, that is, the number of frames.

[0081] When configuring the weight, the effect of the camera's shooting frame rate (fps) on the weight is taken into account. That is, the greater the camera's shooting frame rate (fps) is when shooting the image frames in the circular queue, the greater the base number A in the weight calculation formula is, and the greater the weight is.

[0082] After configuring a weight for each image frame in the circular queue according to the shooting frame rate of the image frames in the circular queue and the frame number interval between each image frame in the circular queue and the current image frame, the weighted sum of the helmet wearing probability of each image frame in the circular queue and the corresponding weight is used as the corrected helmet wearing probability of the current image frame in the circular queue. That is, the weighted sum of the helmet wearing probability of each image frame in the circular queue and the corresponding weight is calculated by formula (1-3), and the weighted sum is the corrected helmet wearing probability of the current image frame.

[0083] In the embodiment of the present application, when configuring the weight for each image frame in the circular queue, the two factors of the shooting frame rate of the image frame in the circular queue and the frame number interval between each image frame in the circular queue and the current image frame are considered. Thus, the minimum weight is configured for the helmet wearing probability P(pre4) corresponding to pre4, which is farthest from the current image frame. And when the shooting frame rate of the camera when shooting the image frame in the circular queue is fps, the larger the frame number interval between the image frames in the circular queue, the greater the influence of the helmet wearing probability of the image frame in the circular queue on the helmet wearing probability of the current image frame. That is, in a very short time, the personnel have no time to take off the helmet or put on the helmet.

[0084] In one embodiment, face detection is performed on the image frames in the circular queue to obtain a face detection result for each image frame in the circular queue, including:

[0085] Performing face detection on the image frames in the circular queue to obtain the face region of each image frame in the circular queue;

[0086] The face area is enlarged by a preset ratio to obtain a target area, and the target area includes the area where the helmet is worn.

[0087] Specifically, a face detection model is used to perform face detection on the image frames in the circular queue to obtain the face area of ​​each image frame in the circular queue. Compared with the traditional method of detecting the helmet area, the method of performing face detection on the image frames in the circular queue using the face detection model has a faster detection speed and higher efficiency. For example, using the face detection model for face detection, it takes about 40ms to process a single frame image on an Intel Core i5 CPU, and it takes about 130ms to process a single frame image on a low-end edge computing processor (such as MTK8665), making it possible to deploy this function on a large scale on low-end edge processors.

[0088] Specifically, the face area can be represented by a rectangular frame formed by the four pixels on the detected face that are farthest in the top, bottom, left, and right directions. The rectangular frame is enlarged by a preset ratio to obtain an enlarged rectangular frame, which is the target area, and the target area includes the area where the helmet is worn. In order to detect that the person is not only wearing a helmet, but also to check whether the helmet is worn correctly, when the rectangular frame is enlarged by a preset ratio to obtain an enlarged rectangular frame, it can be expanded around the rectangular frame corresponding to the face area so that it includes not only the main area of ​​the helmet, but also the hat strap area.

[0089] like Figure 4A and Figure 4B As shown in the figure, the small white rectangular frame in the middle of the face is the rectangular frame of the face area, and the gray rectangular frame set outside the small white rectangular frame is the main area of ​​the helmet formed by expanding to the surrounding area, including the rectangular frame of the hat strap area. Figure 4B As shown in the figure, if the person wears the helmet correctly, Figure 4B The gray rectangular frame includes the main body of the helmet and the strap area. Figure 4A If the person is not wearing a helmet, Figure 4B The main body of the helmet and the chin strap area are not detected in the gray rectangular frame.

[0090] When the rectangular frame is enlarged by a preset ratio, it is necessary to configure the preset ratio with reference to the ratio of the face area to the helmet area in the image frame and the shooting angle of the image frame. Of course, other parameters can also be referred to, and this application does not limit this. For example, when shooting directly, the ratio of the face area to the helmet area is approximately 2:1. At this time, after obtaining the rectangular frame of the face area, the rectangular frame is enlarged upward by 1 / 2, the rectangular frame is enlarged downward by 1 / 4, the rectangular frame is enlarged to the left by 1 / 4, and the rectangular frame is enlarged to the right by 1 / 4. Because the main area of ​​the helmet occupies a larger area, and the proportion of the chin strap area is smaller, the proportion of expansion to the chin strap area is smaller. Of course, this application does not limit the above values.

[0091] When shooting from an upward angle (the camera is located in front of the human body), it is assumed that the ratio of the face area to the helmet area is about 5:1. After obtaining the rectangular frame of the face area, the rectangular frame is expanded upward by 1 / 5, the rectangular frame is expanded downward by 1 / 4, the rectangular frame is expanded to the left by 1 / 4, and the rectangular frame is expanded to the right by 1 / 4. Of course, this application does not limit the above values.

[0092] In the embodiment of the present application, face detection is performed on the image frames in the circular queue to obtain a rectangular frame of the face area of ​​each image frame in the circular queue. The rectangular frame of the face area is then enlarged by a preset ratio to obtain a target area, so that the target area includes not only the area where the helmet is worn, but also the hat strap area. In this way, after ensuring that the target area is completely acquired, it paves the way for subsequent image recognition to determine whether the helmet is worn correctly, thereby improving the accuracy of subsequent determinations of whether the helmet is worn and whether the helmet is worn correctly.

[0093] In one embodiment, the generation process of the preset classifier model includes:

[0094] Inputting a training set into an initialized lightweight deep neural network model for training, wherein the training set includes less than a second preset number of image frames of images of people wearing and not wearing a helmet;

[0095] The parameters of the initialized lightweight deep neural network model are adjusted through the loss function to obtain the preset classifier model.

[0096] Specifically, the lightweight deep neural network model used in this application is the SqueezeNet model, which requires only a small amount of data as a training set, and the training set is input into the initialized SqueezeNet model for training. The loss value is calculated according to the loss function, and the loss value is converged by the gradient descent method to adjust the parameters of the initialized SqueezeNet model to obtain a preset classifier model. The training set includes less than a second preset number of image frames with and without helmets. The second preset number can be the number of image frames that can train a preset classifier model. For example, the second preset number can be configured to 1001. Of course, this application does not limit this value. In short, training a preset classifier model through the initialized SqueezeNet model does not require tens of thousands of image frames as a training set.

[0097] In the embodiment of the present application, the lightweight deep neural network model used is the SqueezeNet model, which only requires less data as a training set. The training set is input into the initialized SqueezeNet model for training to obtain a preset classifier model. Because only less data is required as a training set, the difficulty of the training process is greatly reduced, resources are saved, and efficiency is improved.

[0098] In one embodiment, Figure 5 As shown, a method for detecting wearing of a helmet is provided, further comprising:

[0099] Step 290: When it is determined that the corrected helmet wearing probability of the current image frame in the circular queue is greater than the first threshold, it is detected that the person in the current image frame in the circular queue is wearing a helmet.

[0100] In an embodiment of the present application, the acquired current image frame and the first preset number of image frames before the current image frame are used as a circular queue. Face detection is performed on the image frames in the circular queue to obtain the face detection result of each image frame in the circular queue, and the face detection result of each image frame in the circular queue is input into a preset classifier model to calculate the helmet wearing probability of each image frame. The helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue. When it is determined that the corrected helmet wearing probability of the current image frame in the circular queue is greater than the first threshold, it is detected whether the person in the current image frame in the circular queue wears a helmet and whether the helmet is worn correctly. The first threshold here can be set according to the actual empirical value, for example, it is set to 0.8. Of course, this application does not limit this value.

[0101] In one embodiment, a method for detecting wearing of a helmet is provided, comprising:

[0102] Determine whether the latest helmet wearing probability of each image frame in the circular queue is greater than a first threshold, and obtain a determination result;

[0103] According to the judgment result, the number of the latest image frames in the circular queue whose probability of wearing a helmet is greater than a first threshold is obtained;

[0104] When the number of the image frames is greater than a preset number threshold, it is detected that the person in the current image frame in the circular queue is wearing a safety helmet.

[0105] Specifically, after the helmet wearing probability of each image frame in the circular queue is calculated by the above method, it is determined whether the latest helmet wearing probability of each image frame in the circular queue is greater than a first threshold. The first threshold here can be set according to an actual empirical value, for example, set to 0.8. Of course, this application does not limit this value.

[0106] After judging each image frame in the circular queue, the number of image frames in the circular queue whose latest probability of wearing a helmet is greater than the first threshold is counted. When the number of image frames counted is greater than the preset number threshold, it is detected that the person in the current image frame in the circular queue is wearing a helmet. For example, it can be determined whether the latest probability of wearing a helmet for each image frame in the circular queue is greater than the first threshold. When it is greater than the first threshold, it means that the probability of wearing a helmet is greater, and the character of whether the image frame is wearing a helmet is set to 1. On the contrary, when it is less than or equal to the first threshold, it means that the probability of wearing a helmet is less, and the character of whether the image frame is wearing a helmet is set to 0. Then, the weighted sum of the characters of whether the image frames in the circular queue are wearing a helmet is calculated. When the weighted sum is greater than the preset number threshold, it is detected that the person in the current image frame in the circular queue is wearing a helmet.

[0107] In the embodiment of the present application, after the helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue, and the corrected helmet wearing probability of the current image frame in the circular queue is obtained, it is determined whether the latest helmet wearing probability of each image frame in the circular queue is greater than a first threshold value to obtain a determination result. Then, the number of image frames in the circular queue whose latest helmet wearing probability is greater than the first threshold value is obtained, and when the number of image frames is greater than a preset number threshold value, it is detected that the person in the current image frame in the circular queue is wearing a helmet.

[0108] When correcting the helmet wearing probability of the current frame, the helmet wearing probability of all image frames in the circular queue is referenced. Then, when judging whether the person in the current image frame in the circular queue is wearing a helmet, the situation of whether each image frame in the circular queue is wearing a helmet based on its own helmet wearing probability is also referenced. In this way, from the two dimensions of calculating the helmet wearing probability of the current frame and judging whether the person in the current image frame in the circular queue is wearing a helmet, the relevant values ​​of all image frames in the circular queue are referenced. Therefore, the accuracy of whether the person in the current image frame is wearing a helmet is further improved.

[0109] In one embodiment, Figure 6 As shown, after obtaining the corrected helmet wearing probability of the current image frame in the circular queue, it also includes:

[0110] Step 620, obtaining the next image frame in the circular queue as the current image frame, and deleting the earliest image frame in the circular queue to update the circular queue;

[0111] Step 640, obtaining the helmet wearing probability of the current image frame in the circular queue;

[0112] Step 660, correcting the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue, to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0113] When calculating the corrected helmet wearing probability of the current image frame for the first time, the circular queue includes the current image frame and the first preset number of image frames before the current image frame. At this time, the helmet wearing probability of each image frame in the circular queue is calculated using a preset classifier model. Then, the helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0114] At this time, the next image frame of the circular queue is obtained as the current image frame, and the earliest image frame in the circular queue is deleted to update the circular queue. In the circular queue, except for the unknown probability of wearing a helmet in the current image frame at this time, the probability of wearing a helmet in other image frames is known. Then the probability of wearing a helmet in the current image frame at this time is calculated by a preset classifier. The probability of wearing a helmet in the previous image frame of the current image frame at this time is the probability after correction.

[0115] Finally, the helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0116] This cycle continues until the probability of wearing a helmet in the last frame of the video is calculated.

[0117] In the embodiment of the present application, while the circular queue is continuously updated, the helmet wearing probability of the current image frame is continuously corrected by the helmet wearing probability of the previous image frame, thereby improving the accuracy of the helmet wearing probability of the current image frame calculated each time. Thus, it is detected whether the person in the current image frame in the circular queue is wearing a helmet and whether the helmet is worn correctly.

[0118] In a specific embodiment, Figure 7 As shown, a method for detecting wearing of a helmet is provided, comprising:

[0119] Step 702, obtaining a video stream of the courier driving the vehicle, and proceeding to step 704;

[0120] Step 704, performing face detection on the image frames in the video stream, and proceeding to step 706;

[0121] Step 706, determining whether there is a face in the image frame in the video stream; if so, proceeding to step 708; if not, returning to step 704;

[0122] Step 708, enlarging the rectangular frame obtained by face detection in the first n frames of image frames according to a preset ratio to obtain an enlarged rectangular frame, and then proceeding to step 710;

[0123] Step 710, input the enlarged rectangular frame into a preset classification algorithm, calculate the helmet wearing probability of the first n image frames, form a circular queue with the first n image frames and the corresponding helmet wearing probability, and enter step 712;

[0124] Step 712, determine whether the circular queue has completed initialization, if so, proceed to step 714; if not, return to step 704;

[0125] Step 714, taking the weighted sum of the helmet wearing probability of each image frame in the circular queue and the corresponding weights as the corrected helmet wearing probability of the current image frame in the circular queue, and proceeding to step 716;

[0126] Step 716, determining whether the helmet wearing probability after correction of the current image frame in the circular queue is greater than a first threshold, and proceeding to step 718;

[0127] Step 718, outputting whether the current image frame is in a state of wearing a helmet or not wearing a helmet, and proceeding to step 720;

[0128] Step 720, obtaining the next image frame in the circular queue as the current image frame, deleting the earliest image frame in the circular queue to update the circular queue, and proceeding to step 722;

[0129] Step 722, obtain the enlarged rectangular frame of the current image frame, input the enlarged rectangular frame into the preset classification algorithm, calculate the helmet wearing probability of the current image frame, return to step 712, and determine whether the circular queue has completed initialization.

[0130] In the embodiment of the present application, while the circular queue is continuously updated, the helmet wearing probability of the current image frame is continuously corrected by the helmet wearing probability of the previous image frame, thereby improving the accuracy of the helmet wearing probability of the current image frame calculated each time. Thus, it is detected whether the person in the current image frame in the circular queue is wearing a helmet and whether the helmet is worn correctly.

[0131] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0132] In one embodiment, Figure 8 As shown, a helmet wearing detection device 800 is provided, comprising: a circular queue acquisition module 820, a face detection module 840, a helmet wearing probability calculation module 860 and a helmet wearing probability correction module 880, wherein:

[0133] A circular queue acquisition module 820 is used to acquire a current image frame and a first preset number of image frames before the current image frame as a circular queue;

[0134] A face detection module 840 is used to perform face detection on the image frames in the circular queue to obtain a face detection result for each image frame in the circular queue;

[0135] A helmet wearing probability calculation module 860 is used to input the face detection result of each image frame in the circular queue into a preset classifier model to calculate the helmet wearing probability of each image frame;

[0136] The helmet wearing probability correction module 880 is used to correct the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0137] In one embodiment, the helmet wearing probability correction module 880 is also used to correct the helmet wearing probability of the current image frame in the circular queue according to the helmet wearing probability of each image frame in the circular queue and their corresponding weights, so as to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

[0138] In one embodiment, the helmet wearing probability correction module 880 is also used to configure a weight for each image frame in the circular queue according to the shooting frame rate of the image frames in the circular queue and the frame number interval between each image frame in the circular queue and the current image frame; and the weighted sum of the helmet wearing probability of each image frame in the circular queue and their respective corresponding weights is used as the corrected helmet wearing probability of the current image frame in the circular queue.

[0139] In one embodiment, the face detection module 840 is also used to perform face detection on the image frames in the circular queue to obtain the face area of ​​each image frame in the circular queue; the face area is enlarged by a preset ratio to obtain a target area, and the target area includes the area where the helmet is worn.

[0140] In one embodiment, Fig. 9 As shown, a safety helmet wearing detection device 800 is provided, which also includes: a preset classifier model generation module 890, which is used to input a training set into an initialized lightweight deep neural network model for training, wherein the training set includes image frames of known wearing and not wearing safety helmets that are less than a second preset number; the parameters of the initialized lightweight deep neural network model are adjusted through a loss function to obtain a preset classifier model.

[0141] In one embodiment, a safety helmet wearing detection device 800 is provided, further comprising: a judgment module for detecting that a person in the current image frame in the circular queue is wearing a safety helmet when it is determined that the corrected safety helmet wearing probability of the current image frame in the circular queue is greater than a first threshold.

[0142] In one embodiment, a safety helmet wearing detection device 800 is provided, which also includes: a circulation module, which is used to obtain the next image frame of the circulation queue as the current image frame, and delete the earliest image frame in the circulation queue to update the circulation queue; obtain the safety helmet wearing probability of the current image frame in the circulation queue; correct the safety helmet wearing probability of the current image frame in the circulation queue according to the safety helmet wearing probability of each image frame in the circulation queue, and obtain the corrected safety helmet wearing probability of the current image frame in the circulation queue.

[0143] The division of the various modules in the above-mentioned safety helmet wearing detection device is only for illustration. In other embodiments, the safety helmet wearing detection device can be divided into different modules as needed to complete all or part of the functions of the above-mentioned safety helmet wearing detection device.

[0144] Fig.10 FIG. 1 is a schematic diagram of the internal structure of an electronic device in one embodiment. Fig.10 As shown, the electronic device includes a processor and a memory connected via a system bus. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement a helmet wearing detection method provided in the following embodiments. The internal memory provides a cached operating environment for the operating system computer program in the non-volatile storage medium. The electronic device may be a mobile phone, a tablet computer, a personal digital assistant, a wearable device, etc.

[0145] The implementation of each module in the helmet wearing detection device provided in the embodiment of the present application can be in the form of a computer program. The computer program can be run on a terminal or a server. The program modules constituted by the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, the steps of the method described in the embodiment of the present application are implemented.

[0146] The embodiment of the present application also provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, enable the processors to perform the steps of the helmet wearing detection method.

[0147] A computer program product comprising instructions, when the computer program product is run on a computer, causes the computer to execute a helmet wearing detection method.

[0148] Any reference to memory, storage, database or other media used in the embodiments of the present application may include non-volatile and / or volatile memory. Suitable non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0149] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting the wearing of a helmet, characterized in that: include: Using the acquired current image frame and a first preset number of image frames before the current image frame as a circular queue; Performing face detection on the image frames in the circular queue to obtain a face region of each image frame in the circular queue; Enlarging the face area by a preset ratio to obtain a target area, wherein the target area includes an area where a helmet is worn; The preset ratio is determined based on the ratio of the face area to the helmet area in the image frame; the ratio of the face area to the helmet area is determined according to the shooting angle of the image frame; Inputting the rectangular frame of the target area of ​​each image frame in the circular queue into a preset classifier model to obtain the helmet wearing probability of each image frame; The weighted sum of the helmet wearing probability of each image frame in the circular queue and their corresponding weights is used as the corrected helmet wearing probability of the current image frame in the circular queue; the weight is obtained based on the shooting frame rate of the image frames in the circular queue and the frame number interval between each image frame in the circular queue and the current image frame; the weight is inversely correlated with the frame number interval between each image frame in the circular queue and the current image frame; the weight is positively correlated with the shooting frame rate of the image frames in the circular queue.

2. The method according to claim 1, characterized in that The generation process of the preset classifier model includes: Inputting a training set into an initialized lightweight deep neural network model for training, wherein the training set includes less than a second preset number of image frames of images of people wearing and not wearing a helmet; The parameters of the initialized lightweight deep neural network model are adjusted through the loss function to obtain the preset classifier model.

3. The method according to claim 1, characterized in that The method further comprises: When it is determined that the corrected helmet wearing probability of the current image frame in the circular queue is greater than the first threshold, it is detected that the person in the current image frame in the circular queue is wearing a helmet.

4. The method according to claim 1, characterized in that: The method further comprises: Determine whether the latest helmet wearing probability of each image frame in the circular queue is greater than a first threshold, and obtain a determination result; According to the judgment result, the number of the latest image frames in the circular queue whose probability of wearing a helmet is greater than a first threshold is obtained; When the number of the image frames is greater than a preset number threshold, it is detected that the person in the current image frame in the circular queue is wearing a safety helmet.

5. The method according to claim 1, characterized in that After obtaining the corrected helmet wearing probability of the current image frame in the circular queue, the method further includes: Acquire the next image frame in the circular queue as the current image frame, and delete the earliest image frame in the circular queue to update the circular queue; Obtaining the helmet wearing probability of the current image frame in the circular queue; The helmet wearing probability of the current image frame in the circular queue is corrected according to the helmet wearing probability of each image frame in the circular queue to obtain the corrected helmet wearing probability of the current image frame in the circular queue.

6. A helmet wearing detection device, characterized in that: include: A circular queue acquisition module, used for acquiring a current image frame and a first preset number of image frames before the current image frame as a circular queue; A face detection module is used to perform face detection on the image frames in the circular queue to obtain a face area of ​​each image frame in the circular queue; the face area is enlarged by a preset ratio to obtain a target area, and the target area includes an area where a helmet is worn; The preset ratio is determined based on the ratio of the face area to the helmet area in the image frame; the ratio of the face area to the helmet area is determined according to the shooting angle of the image frame; A helmet wearing probability calculation module, used for inputting a rectangular frame of a target area of ​​each image frame in the circular queue into a preset classifier model to calculate a helmet wearing probability of each image frame; A helmet wearing probability correction module is used to take the weighted sum of the helmet wearing probability of each image frame in the circular queue and their respective corresponding weights as the corrected helmet wearing probability of the current image frame in the circular queue; the weight is obtained based on the shooting frame rate of the image frames in the circular queue and the frame number interval between each image frame in the circular queue and the current image frame; the weight is inversely correlated with the frame number interval between each image frame in the circular queue and the current image frame; the weight is positively correlated with the shooting frame rate of the image frames in the circular queue.

7. The device according to claim 6, characterized in that The device also includes a preset classifier model generation module, which is used to input a training set into an initialized lightweight deep neural network model for training, wherein the training set includes less than a second preset number of image frames of known wearing and not wearing a safety helmet; and adjust the parameters of the initialized lightweight deep neural network model through a loss function to obtain a preset classifier model.

8. The device according to claim 6, characterized in that The device also includes a judgment module for detecting that the person in the current image frame in the circular queue is wearing a safety helmet when it is judged that the corrected safety helmet wearing probability of the current image frame in the circular queue is greater than a first threshold.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor executes the steps of the helmet wearing detection method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the helmet wearing detection method according to any one of claims 1 to 5 are implemented.

Images