A smoking identification method, system, device and storage medium

By performing facial area extraction and super-resolution enhancement processing in smoking recognition, combined with the yolov5 model and channel space attention module, the problem of low accuracy of smoking recognition in the prior art is solved, and the recall rate of detection is improved and the reduction of detection is The false detection rate achieves efficient smoking identification.

CN114944009BActive Publication Date: 2024-11-15CHINA UNICOM (GUANGDONG) IND INTERNET CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202210508435.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-10
Publication Date
2024-11-15
Estimated Expiration
2042-05-10

AI Technical Summary

Technical Problem

The prior art has low accuracy in smoking recognition, especially when the proportion of the smoking part target is small and unclear, resulting in high rates of false detection and missed detection, and it is impossible to effectively monitor smoking behavior in smokeless areas.

Method used

Using an improved smoking recognition method, by obtaining the image to be detected, performing facial area extraction and super-resolution enhancement processing, smoking detection is performed using the yolov5 model framework, and a parallel channel space is added between the skeleton feature extraction network and the neck network. The force module combines the generative adversarial network ESRGAN module and the channel space attention module to improve the accuracy of detection.

Benefits of technology

It effectively improves the recall rate of smoking detection and reduces the false detection rate, provides high-quality and high-resolution input pictures, improves the accuracy of smoking recognition, and accelerates the inference speed of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114944009B_ABST
    Figure CN114944009B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image processing, and more specifically, to a smoking recognition method, system, device and storage medium. The method comprises: obtaining an image to be detected; processing the image to be detected to extract a face area; performing smoking detection on the face area using a yolov5 model framework to obtain a smoking detection result; wherein the backbone of the skeleton feature extraction network and the neck network neck in the yolov5 model framework are connected by a plurality of parallel channel space attention modules. The smoking recognition of the present invention first performs face detection, and then inputs it into yolov5 with an attention mechanism module added to perform smoking detection recognition. The recall rate of smoking detection can be effectively improved by cascading detection, while the false detection rate is reduced to a certain extent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically, to a smoking recognition method, system, device and storage medium. Background Art

[0002] Among the health problems faced by humans, smoking is undoubtedly the most harmful problem recognized by the world. It not only harms one's own physical and mental health, but also has a greater health impact on people who inhale secondhand smoke, directly or indirectly leading to the occurrence of many diseases. In poorly ventilated office areas, employees are forced to inhale secondhand smoke, which has a great impact on their health, and this situation is particularly serious. In addition, smoking may also cause fires, resulting in significant property losses and casualties.

[0003] If we only rely on manual monitoring of videos to find smokers, this method is time-consuming and laborious. At the same time, the amount of video data is large and the monitoring time is long, which can easily cause fatigue to personnel and make it impossible to make timely and accurate judgments on abnormal situations. Therefore, it is necessary to use monitoring technology to detect smoking in non-smoking areas to reduce the waste of manpower and material resources. At this time, smoking recognition and detection technology based on computer vision is particularly important.

[0004] Traditional feature extraction based on image segmentation and manually designed algorithms can detect smoking behavior by detecting smoke, but the accuracy is low and can only detect smoking in specific situations. With the development of deep learning, smoking recognition algorithms based on target detection have developed rapidly, but some detections are directly based on the entire image for smoking detection, and the proportion of targets in the smoke is relatively small, and there are unclear situations, resulting in a large number of false detections and missed detections. Summary of the invention

[0005] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a smoking recognition method, system, device and storage medium for improving the accuracy of smoking recognition.

[0006] The technical solution adopted by the present invention is a smoking identification method, which comprises:

[0007] Acquire the image to be detected;

[0008] Process the image to be detected to extract the face area;

[0009] Use the yolov5 model framework to perform smoking detection on the face area to obtain the smoking detection result;

[0010] Among them, the skeleton feature extraction network backbone and the neck network neck in the yolov5 model framework are connected through several parallel channel space attention modules.

[0011] Furthermore, after the face region is extracted, the image to be detected is also subjected to super-resolution enhancement processing, and then input into the yolov5 target detection model for smoking detection after the super-resolution enhancement processing.

[0012] Furthermore, the super-resolution enhancement processing is implemented based on the generative adversarial network ESRGAN module.

[0013] Furthermore, the generative adversarial network ESRGAN module includes a first RRDB module group, a first convolutional layer, a second RRDB module group, a second convolutional layer, an upsample module, a third convolutional layer, and a fourth convolutional layer connected in sequence, and a shortcut connection is adopted between the first RRDB module group and the second RRDB module.

[0014] Furthermore, the first RRDB module group and the second RRDB module group respectively include 8 RRDB modules connected in sequence and a fifth convolutional layer.

[0015] Furthermore, the backbone of the skeleton feature extraction network includes the sixth convolutional layer, the seventh convolutional layer, three C3 modules, the eighth convolutional layer, six C3 modules, the ninth convolutional layer, nine C3 modules, the tenth convolutional layer, three C3 modules and the SPPF module, which are connected in sequence, wherein the six C3 modules, the nine C3 modules and the SPPF module are connected to the neck network through a channel spatial attention module respectively;

[0016] The C3 module includes two branches. The first branch includes the eleventh convolutional layer and the Bottleneck module connected in sequence, and the second branch includes the twelfth convolutional layer. The two branches enter the thirteenth convolutional layer after tensor concat.

[0017] Furthermore, the channel spatial attention module includes a spatial attention module and a channel attention module connected in parallel, and the input features are processed in the spatial attention module and the channel attention module respectively and then fused and outputted.

[0018] A smoking identification system, the system comprising:

[0019] An image acquisition module, used for acquiring an image to be detected;

[0020] A face region extraction module is used to process the image to be detected and extract the face region;

[0021] The detection module is used to detect smoking in the face area using the yolov5 model framework to obtain the smoking detection result;

[0022] Among them, the skeleton feature extraction network backbone and the neck network neck in the yolov5 model framework are connected through several parallel channel space attention modules.

[0023] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned smoking identification method when executing the computer program.

[0024] A computer-readable storage medium stores a computer program, wherein the computer program implements the smoking identification method as described above when executed by a processor.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] (1) The smoking recognition method and system of the present invention first perform face detection, then use the improved super-resolution module to magnify and enhance the face image, and finally input it into yolov5 with an attention mechanism module added to perform smoking detection recognition. The cascade detection method can effectively improve the recall rate of smoking detection, while reducing the false detection rate to a certain extent.

[0027] (2) The present invention enhances the image of the face part by super-resolution enhancement, which increases the original resolution by a certain multiple, and can provide high-quality and high-resolution input images for subsequent smoking recognition, thereby better extracting image features.

[0028] (3) The super-resolution enhancement processing in the present invention is based on the generative adversarial network ESRGAN, which divides the original stacked RRDB module into two stages. Through the shortcut connection method, the network can be better trained and learned. At the same time, the RRDB module is reduced, making the network lighter and accelerating the reasoning speed of the model.

[0029] (4) The present invention adds a parallel channel spatial self-attention module CSSA (Channel Spatial Self-Attention) between the backbone and neck of the yolov5 model. In this module, the channel attention and spatial attention are combined in parallel. Compared with the serial attention mechanism module, the calculation speed is faster, the useful information of the image can be better extracted, and the detection rate of small targets is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The present invention is a flow chart of a smoking identification method.

[0031] Figure 2 This is the improved ESRGAN module architecture diagram of the present invention.

[0032] Figure 3 This is an architectural diagram of the RRDB module in the present invention.

[0033] Figure 4 This is the architecture diagram of the improved yolov5 in the present invention.

[0034] Figure 5 This is the architecture diagram of the channel space attention module CSSA in the present invention.

[0035] Figure 6 This is a structural diagram of a smoking identification system in the present invention. DETAILED DESCRIPTION

[0036] The drawings of the present invention are only for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0037] Example 1

[0038] like Figure 1 As shown in FIG. 1 , a flow chart of a smoking identification method of this embodiment is shown. Figure 1 The specific steps of a smoking identification method in this embodiment include:

[0039] S101. Obtaining an image to be detected;

[0040] S102. Processing the image to be detected to extract the face area;

[0041] In order to improve the recognition accuracy, the face of the image to be detected is first detected, the face position is located, and then the smoke recognition is performed in the face area. In the specific implementation process, the face detection algorithm such as centerface can be used.

[0042] S103. The face region of the image to be detected is extracted and then subjected to super-resolution enhancement processing;

[0043] S104. Using the yolov5 model framework, smoke detection is performed on the face area after super-resolution enhancement to obtain a smoke detection result;

[0044] Among them, the skeleton feature extraction network backbone and the neck network neck in the yolov5 model framework are connected through several parallel channel space attention modules.

[0045] In this specific embodiment, the face area of ​​the image to be detected is first extracted, because the act of smoking is man-made. Therefore, the face is first located, which can better locate the smoking behavior. Secondly, the extracted face area is subjected to super-resolution enhancement processing, which can improve the texture information of the face image. When there is smoking behavior in the face image, the clarity of the smoke is also enhanced, which helps to improve the accuracy of subsequent smoking recognition; finally, this specific embodiment uses a combination of channel attention and spatial attention to improve the yolov5 model, which can make the features more effectively focus on the parts of the object to be identified, and the probability of finally distinguishing the object is also higher, which improves the accuracy of smoking recognition.

[0046] In the specific implementation process, Figure 2 As shown, the super-resolution enhancement process is implemented based on the generative adversarial network ESRGAN module, and the ESRGAN module is improved, specifically:

[0047] The ESRGAN module based on the generative adversarial network includes a first RRDB module group, a first convolutional layer (Conv), a second RRDB module group, a second convolutional layer (Conv), an upsample module, a third convolutional layer (Conv), and a fourth convolutional layer (Conv) connected in sequence, and a shortcut connection is adopted between the first RRDB module group and the second RRDB module;

[0048] The first RRDB module group and the second RRDB module group respectively include 8 RRDB modules connected in sequence and a fifth convolutional layer (Conv), and the composition of the RRDB modules is as follows: Figure 3 shown.

[0049] In security monitoring scenarios, the camera resolution usually used is 1080P. The size of smoke is relatively small, and it is also affected by various lighting conditions on site, making the clarity of the captured smoke even more blurred. Based on this situation, before performing smoke recognition, super-resolution enhancement of the image of the face area can improve the recognition rate.

[0050] The existing ESRGAN module is composed of 23 RRDB modules in series before entering the upsample module. Considering the better convergence of the network and reducing the amount of calculation, this specific embodiment modifies the 23 RRDB modules in the prior art into a two-stage series connection, with 8 RRDB modules in each stage, and a shortcut connection method is used to accelerate the convergence of the network. The present invention can magnify the resolution of the detected face image by 4 times after passing it through the improved generative adversarial network ESRGAN, improve the texture information of the face image, and when there is smoking in the face image, it also enhances the clarity of the smoke, which helps to improve the accuracy of subsequent smoking detection.

[0051] In the specific implementation process, the parallel channel spatial attention module includes a spatial attention module and a channel attention module connected in parallel with each other. The input features are processed in the spatial attention module and the channel attention module respectively and then fused and output.

[0052] In the channel self-attention module, each channel of the feature represents a dedicated detector, which enables the model to pay more attention to useful information and play an enhancement role, while weakening irrelevant channel information, allowing the network to pay more attention to useful channel information.

[0053] The spatial self-attention module focuses on which features have stronger representative significance. Figure 5 As shown, this specific embodiment combines the channel self-attention module (Channel Self-Attention) and the spatial self-attention module (SpatialSelf-Attention) in parallel, which is faster to calculate than the series method. At the same time, it can also allow the features to focus more effectively on the parts of the object to be identified, and the probability of ultimately distinguishing the object is also higher, thereby improving the detection rate of smoking.

[0054] Specifically, Figure 4 As shown, the backbone of the skeleton feature extraction network includes the sixth convolution layer, the seventh convolution layer, three serially connected C3 modules, the eighth convolution layer, six serially connected C3 modules, the ninth convolution layer, nine serially connected C3 modules, the tenth convolution layer, three serially connected C3 modules and the SPPF module, wherein the six serially connected C3 modules, the nine serially connected C3 modules and the SPPF module are respectively connected to the neck network through a parallel channel space attention module CSSA;

[0055] The C3 module includes two branches. The first branch includes the eleventh convolutional layer and the Bottleneck module connected in sequence, and the second branch includes the twelfth convolutional layer. The two branches enter the thirteenth convolutional layer after tensor concat.

[0056] The smoking recognition method of the present invention first performs face detection, then uses the improved super-resolution module to amplify and enhance the face image, and finally inputs it into yolov5 with an attention mechanism module added to perform smoking detection recognition. The cascade detection method can effectively improve the recall rate of smoking detection, while reducing the false detection rate to a certain extent and improving the smoking recognition rate.

[0057] Example 2

[0058] like Figure 6 As shown, this specific embodiment provides a smoking identification system, the system comprising:

[0059] An image acquisition module 201 is used to acquire an image to be detected;

[0060] A face region extraction module 202, used to process the image to be detected and extract the face region;

[0061] In order to improve the recognition accuracy, the face region extraction module 202 is used to first detect the face of the image to be detected, locate the face position, and then perform smoke recognition in the face region. In the specific implementation process, a face detection algorithm such as centerface can be used.

[0062] A super-resolution enhancement module 203, used for performing super-resolution enhancement processing after extracting the face region of the image to be detected;

[0063] The detection module 204 is used to perform smoking detection on the face area using the yolov5 model framework to obtain a smoking detection result;

[0064] Among them, the skeleton feature extraction network backbone and the neck network neck in the yolov5 model framework are connected through several parallel channel space attention modules.

[0065] In this specific embodiment, the face region extraction module 202 is used to extract the face region of the image to be detected first, because the act of smoking is man-made, therefore, the face is located first, so that the smoking behavior can be better located. Secondly, the super-resolution enhancement module 203 is used to perform super-resolution enhancement processing on the extracted face region, which can improve the texture information of the face image. When there is smoking behavior in the face image, the clarity of the smoke is also enhanced, which helps to improve the accuracy of subsequent smoking recognition; finally, this specific embodiment uses the detection module 204 to improve the yolov5 model based on the combination of channel attention and spatial attention, which can make the features more effectively focus on the parts of the object to be identified, and the probability of finally distinguishing the object is also higher, which improves the accuracy of smoking recognition.

[0066] In the specific implementation process, the super-resolution enhancement processing of the super-resolution enhancement module 203 is implemented based on the generative adversarial network ESRGAN module, and the ESRGAN module is improved, such as Figure 2 As shown, specifically:

[0067] The ESRGAN module based on the generative adversarial network includes a first RRDB module group, a first convolutional layer (Conv), a second RRDB module group, a second convolutional layer (Conv), an upsample module, a third convolutional layer (Conv), and a fourth convolutional layer (Conv) connected in sequence, and a shortcut connection is adopted between the first RRDB module group and the second RRDB module;

[0068] The first RRDB module group and the second RRDB module group respectively include 8 RRDB modules connected in sequence and a fifth convolutional layer (Conv), and the composition of the RRDB modules is as follows: Figure 3 shown.

[0069] In security monitoring scenarios, the camera resolution usually used is 1080P. The size of smoke is relatively small, and it is also affected by various lighting conditions on site, making the clarity of the smoke even more blurred. Based on this situation, before performing smoke recognition, super-resolution enhancement of the image of the face area can improve the recognition rate.

[0070] The existing ESRGAN module is composed of 23 RRDB modules in series before entering the upsample module. Considering the better convergence of the network and reducing the amount of calculation, this specific embodiment modifies the 23 RRDB modules into a two-stage series connection, with 8 RRDB modules in each stage, and a shortcut connection method is used to accelerate the convergence of the network. The present invention can magnify the resolution of the detected face image by 4 times after passing it through the improved generative adversarial network ESRGAN, improve the texture information of the face image, and when there is smoking in the face image, it also enhances the clarity of the smoke, which helps to improve the accuracy of subsequent smoking detection.

[0071] In the specific implementation process, the channel spatial attention module in the detection module 204 includes a spatial attention module and a channel attention module connected in parallel, and the input features are respectively processed in the spatial attention module and the channel attention module and then fused and output.

[0072] In the channel self-attention module, each channel of the feature represents a dedicated detector, which enables the model to pay more attention to useful information and play an enhancement role, while weakening irrelevant channel information, allowing the network to pay more attention to useful channel information.

[0073] The spatial self-attention module focuses on where the features have stronger representational significance.

[0074] like Figure 5 As shown, this specific embodiment combines the channel self-attention module (Channel Self-Attention) and the spatial self-attention module (Spatial Self-Attention) in parallel, which is faster to calculate than the series method. At the same time, it can also allow the features to focus more effectively on the parts of the object to be identified, and the probability of ultimately distinguishing the object is also higher, thereby improving the detection rate of smoking.

[0075] Specifically, Figure 4 As shown, the backbone of the skeleton feature extraction network includes the sixth convolution layer, the seventh convolution layer, three series-connected C3 modules, the eighth convolution layer, six series-connected C3 modules, the ninth convolution layer, nine series-connected C3 modules, the tenth convolution layer, three series-connected C3 modules and the SPPF module, wherein the six series-connected C3 modules, the nine series-connected C3 modules and the SPPF module are respectively connected to the neck network through a channel space attention module CSSA;

[0076] The C3 module includes two branches. The first branch includes the eleventh convolutional layer and the Bottleneck module connected in sequence, and the second branch includes the twelfth convolutional layer. The two branches enter the thirteenth convolutional layer after tensor splicing Conact.

[0077] The smoking recognition system of the present invention first performs face detection, and then uses the super-resolution enhancement module 203 to magnify and enhance the face image using the improved super-resolution module, and finally inputs it into the detection module 204 to which the attention mechanism module is added to perform smoking detection recognition. The cascade detection method can effectively improve the recall rate of smoking detection, while reducing the false detection rate to a certain extent, thereby improving the recognition rate of smoking.

[0078] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A smoking identification method, It is characterized in that The method comprises: Acquire the image to be detected; Process the image to be detected to extract the face area; Use the yolov5 model framework to perform smoking detection on the face area to obtain the smoking detection result; Among them, the skeleton feature extraction network backbone and the neck network neck in the yolov5 model framework are connected through several parallel channel space attention modules; After the face region is extracted, the image to be detected is subjected to super-resolution enhancement processing, and then input into the yolov5 target detection model for smoking detection after the super-resolution enhancement processing; The super-resolution enhancement process is implemented based on the generative adversarial network ESRGAN module; The ESRGAN module based on the generative adversarial network includes a first RRDB module group, a first convolutional layer, a second RRDB module group, a second convolutional layer, an upsample module, a third convolutional layer, and a fourth convolutional layer connected in sequence, and a shortcut connection is adopted between the first RRDB module group and the second RRDB module; The first RRDB module group and the second RRDB module group respectively include 8 RRDB modules and a fifth convolutional layer connected in sequence; The channel spatial attention module includes a spatial attention module and a channel attention module connected in parallel. The input features are processed by the spatial attention module and the channel attention module respectively and then fused and output.

2. The smoking identification method according to claim 1, It is characterized in that The backbone network for skeleton feature extraction includes the sixth convolutional layer, the seventh convolutional layer, three C3 modules, the eighth convolutional layer, six C3 modules, the ninth convolutional layer, nine C3 modules, the tenth convolutional layer, three C3 modules and the SPPF module, which are connected in sequence. The six C3 modules, the nine C3 modules and the SPPF module are connected to the neck network through a channel spatial attention module respectively. The C3 module includes two branches. The first branch includes the eleventh convolutional layer and the Bottleneck module connected in sequence, and the second branch includes the twelfth convolutional layer. The two branches enter the thirteenth convolutional layer after tensor concat.

3. A smoking recognition system, It is characterized in that The system comprises: An image acquisition module, used for acquiring an image to be detected; A face region extraction module is used to process the image to be detected and extract the face region; The detection module is used to detect smoking in the face area using the yolov5 model framework to obtain the smoking detection result; Among them, the skeleton feature extraction network backbone and the neck network neck in the yolov5 model framework are connected through several parallel channel space attention modules; A super-resolution enhancement module is used to perform super-resolution enhancement processing after extracting the face area of ​​the image to be detected; Among them, the super-resolution enhancement module is composed of a generative adversarial network ESRGAN module, and the generative adversarial network ESRGAN module includes a first RRDB module group, a first convolutional layer, a second RRDB module group, a second convolutional layer, an upsample module, a third convolutional layer, and a fourth convolutional layer connected in sequence, and a shortcut connection is adopted between the first RRDB module group and the second RRDB module; The first RRDB module group and the second RRDB module group respectively include 8 RRDB modules and a fifth convolutional layer connected in sequence; The channel spatial attention module includes a spatial attention module and a channel attention module connected in parallel, which are used to input features into the spatial attention module and the channel attention module respectively for processing and then perform fusion output.

4. A computer device comprising a memory and a processor, the memory storing a computer program, It is characterized in that When the processor executes the computer program, the smoking identification method according to any one of claims 1 to 2 is implemented.

5. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the smoking recognition method according to any one of claims 1 to 2 is implemented.

Citation Information

Patent Citations

  • Smoking behavior detection method and device

    CN112380977A

  • Smoking behavior identification method based on optimized YOLOv4 model

    CN113807276A

  • Coal gangue target detection method based on improved YOLOv5 algorithm

    CN114120093A

  • Student classroom behavior detection method based on ESRGAN and improved YOLOv5s

    CN114241422A

  • Dangerous driving behavior recognition method and apparatus, and electronic device and storage medium

    WO2022001091A1