A method and system for infrared image smoking detection based on deep learning

Abstract

The present invention discloses a method and system for detecting smoking in an infrared image based on deep learning, relates to the technical field of image recognition, and solves the technical problems of unstable smoke point detection accuracy and poor positioning effect in existing infrared scenes, including mounting hardware environment, data collection, data set preprocessing and production, smoke point feature extraction, smoke point feature fusion, smoke point detection and smoke point alarm. The present invention takes a dual-camera of infrared and visible light as the hardware basis, fuses visible light with infrared images, combines a smoking behavior detection method based on deep learning, and enriches sample semantic features and positioning features through an FPN+PAN structure; uses image adaptive scaling and filling to reduce the computational cost, reduce the deployment hardware requirements, improve the model recognition accuracy and real-time performance, and effectively realizes smoking recognition and detection in infrared scenes; forms a smoking behavior detection system that can be deployed in factories, workshops and other workplaces to perform effective and stable smoking behavior detection.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and more specifically to the field of infrared image smoking detection method and system technology based on deep learning. Background Art

[0002] my country is the only country in the world with a complete industrial system. Factory safety protection is an important guarantee for the development of the industrial economy. According to incomplete statistics, fire is one of the most frequent and serious problems in factory safety accidents, and smoking is one of the main factors causing factory fires. Therefore, smoking detection in the factory environment is an important means to ensure factory safety. Smoking detection methods are mainly divided into traditional methods, detection methods based on smoke recognition, and methods based on convolutional neural networks:

[0003] 1. Traditional methods: In 2010, Wu Pin et al. extracted the features between a lit cigarette and its smoker by using a feature extraction method based on color histogram, combined with Gaussian mixture model technology to achieve the task of cigarette segmentation and tracking background pixels, and detected cigarette targets of various sizes, colors and shapes; In 2011, Wu Wen-C et al. proposed a system for detecting smoking behavior through faces; First, the existing face detection scheme is used to locate the mouth of a person, and then the face image is white-balanced to reduce the misjudgment caused by light, and color segmentation and noise elimination are implemented in the HSV color system. The connection of the region after binary morphological expansion and erosion of the object, that is, the range between the cigarette and the mouth, is used to judge whether there is smoking behavior. However, due to the incorrect facial posture and complex image background, the face detection effect is not good; In 2013, Wang Chao et al. proposed a "segmentation first and then recognition" smoking gesture recognition algorithm for the specific behavior characteristics of smoking gestures. The gesture area is first segmented by combining the motion information of the video sequence with skin color detection, and then the support vector machine is used to detect smoking gestures by extracting gesture features.

[0004] Considering that color features are greatly affected by ambient lighting and cigarette targets are small in size, it is relatively difficult to extract color from complex backgrounds. In addition, suspected cigarette objects will affect the judgment of cigarette recognition, which can easily lead to misjudgment. In addition, smoking gestures are diverse, and the accuracy of detecting smoking behavior through smoking postures is not high. Therefore, researchers have tried to detect smoking through the smoke produced by smoking.

[0005] 2. Detection method based on smoke recognition: In 2013, Ding Hongjie studied the detection of smoking behavior based on smoke recognition in indoor places. He used a combination of background difference method and color model to segment the suspected area of ​​cigarette smoke, and then extracted the cigarette smoke features according to the smoke primitive block and spatiotemporal characteristics. Finally, he realized the cigarette smoke recognition based on SVM to achieve the purpose of detecting smoking; in 2014, Su Xiangyu proposed a multi-feature smoking behavior detection algorithm that first determines the spatial interest range through the relative position of the human hand and face, and then classifies and judges the suspected smoke area according to the dynamic characteristics of cigarette smoke; in 2016, Ai Bo scholar used a mixed Gaussian based on the problem that cigarette smoke is relatively thin and difficult to distinguish the background. The model background modeling method updates the background of the monitored area in real time, and extracts the foreground target in the video frame by combining the background subtraction method. After the geometric features of the area of ​​interest are preliminarily judged, the classifier is used to identify cigarette smoke, which greatly improves the accuracy of smoking behavior detection. In 2018, Liu Yuanding et al. analyzed and extracted the HOG features and texture features of cigarette smoke, and applied the feature fusion algorithm to realize the recognition of the smoke emitted when smoking. In the same year, Xing Yingbiao, Huang Haitao and other scholars proposed a smoking behavior detection algorithm. First, cigarettes are identified based on the number of white pixels on the located facial image, and then the smoke in the facial area is identified according to the change of the energy value of the facial area image during the detection process. Finally, the smoking behavior is judged based on the identified cigarettes and smoke. Because the smoke concentration of cigarettes is low, it is easy to diffuse and has unclear edges, which causes many difficulties for target detection, resulting in the fact that the effect of smoke-based detection is not very good. With the continuous development of deep learning, the robustness of neural network models has gradually improved. Traditional methods are subject to certain limitations in feature extraction. Deep learning can better extract the semantic information of smoking behavior and improve the accuracy of smoking detection.

[0006] 3. Methods based on convolutional neural networks: In 2017, for driver smoking detection, Artan Y et al. proposed a method for detecting driver smoking behavior using near-infrared (NIR) surveillance camera images. First, the deep learning-based target detection technology was used to locate the front windshield and the driver's head area in turn. Then, a dual-window (local) anomaly detector was performed in the local area to determine the white hot spots caused by high temperature on the NIR image, thereby determining the driver's smoking behavior. This method is only applied to local small spaces and has not been tried to be expanded to larger indoor spaces for detection. In 2019, Zhao R et al. proposed an indoor smoking behavior detection algorithm based on the YOLOv3-tiny deep learning network. The k-means clustering algorithm was used to obtain the prior frame of the cigarette target. A small target detection layer was added to the original YOLOv3-tiny network, which effectively met the actual application needs and provided a new way to assist indoor monitoring. However, there is still room for optimization. In 2020, Chen Ruilong et al. proposed a real-time smoking detection method based on deep learning. The model uses a convolutional neural network to process the video stream input captured by the camera. After image feature extraction, feature fusion, target classification and target positioning, the position of cigarette butts is located, and then the smoking behavior is judged. The common target detection algorithm is not ideal for small target detection, and the detection speed needs to be improved; through the design of a series of convolutional neural network modules, not only the model calculation amount is reduced, the deduction speed is accelerated, and the real-time requirements are met, but also the accuracy of small target (cigarette butt) detection is improved. This method optimizes the structure of the neural network many times and performs well in feature extraction. Therefore, the use of smoking detection based on deep learning can greatly improve the detection accuracy, enhance the effectiveness of fire warning, and provide effective protection for social economy and people's life safety.

[0007] The smoking process is accompanied by temperature changes in infrared images. The smoke point radiates from the highest temperature of the cigarette butt to the surrounding area to form a nearly circular color temperature spot, and the color changes from dark to light from the inside to the outside; In addition, with the influence of factors such as scene depth, seasonal temperature difference, and day and night illumination, the smoke point representation is diverse. Therefore, there are great limitations in using traditional methods to detect smoke points. Deep learning has strong learning ability, and can estimate the rules of samples through data support, improve the robustness of the network model, and achieve high-accuracy smoke point detection; Therefore, using a deep learning-based detection method to achieve smoke point detection is a reasonable method. From the above, it can be seen that detecting smoke points in an infrared environment can effectively determine whether there is smoking behavior. However, infrared images often have the characteristics of poor resolution, low contrast, low signal-to-noise ratio, blurred visual effects, and more monotonous information compared to visible light images. Therefore, effective and stable target detection cannot be obtained in infrared scenes. Due to the characteristics of poor resolution, low contrast, low signal-to-noise ratio, blurred visual effects, and more monotonous information in infrared images compared to visible light images, effective and stable target detection cannot be achieved in infrared scenes.

[0008] In summary, the existing technology still has the following defects: 1) The target feature information in infrared images is not rich and lacks contextual information, and the target detection accuracy of infrared images is poor; 2) The area of ​​smoke points in the sample is small, sometimes there are only a few pixels, and the detection and positioning effect of small targets is poor; 3) The traditional method has great limitations. When the smoke points are affected by environmental factors and present different forms in the sample, resulting in a large class distance, the traditional method cannot effectively detect the smoke points. Summary of the invention

[0009] The purpose of the present invention is to solve the above-mentioned technical problems and provide an infrared image smoking detection method and system based on deep learning.

[0010] The technical solution adopted by the present invention is as follows: a method for detecting smoking in infrared images based on deep learning, comprising the following steps:

[0011] Step 1: Install the hardware environment and install the infrared and visible light binocular cameras according to the security camera installation standards, and electrically connect them to the terminal processor respectively;

[0012] Step 2: Data collection: Use an infrared visible binocular camera to shoot a video of a simulated workplace, collect smoking behavior videos of three types of simulated real smoking scenes: single person, double person, and multiple person, and obtain visible light video and infrared video at the same time, and take frames at the same time starting point and frequency to obtain visible light initial samples and infrared initial samples;

[0013] Step 3: Data preprocessing: performing image fusion processing on the visible light initial sample and the infrared initial sample to obtain a fused sample;

[0014] Step 4: Dataset creation: Filter the fused samples to obtain 2,700 images and annotate them. The samples are dual-light fusion images with a size of 640×480×3. The 2,700 annotated samples are divided into training set, test set and validation set in a ratio of 8:1:1.

[0015] Step 5: Extract smoke point features. Use the Yolov5 network model to perform Mosaic enhancement on the input samples. After the original samples are adaptively scaled and filled, the input sample size becomes 640×640×3. Input it into the backbone network, slice the sample into 320×320×12 through the Focus structure, and then perform a convolution operation with 64 convolution kernels to obtain a feature map. The backbone network completes the smoke point feature extraction.

[0016] Step 6: Fusion of smoke point features. Yolov5 uses the FPN+PAN structure to process the backbone network to extract the smoke point features.

[0017] Step 7: Detect smoke points. Use non-maximum suppression (NMS) to filter the target frame and retain the prediction frame with the highest confidence. This completes the smoke point detection process and determines that images with smoke points are smoked.

[0018] Step 8: Smoke point alarm. Deploy the smoke point detection model to the terminal processor. Transmit the video captured by the infrared and visible light binocular camera to the terminal processor via Ethernet. Implement real-time smoke point monitoring within the camera shooting range. Once a smoke point is detected, send a command to the alarm device via the serial port to complete the alarm.

[0019] The working principle of the present invention is as follows: the present invention uses an infrared and visible light dual-camera as the hardware basis, collects smoking behavior images through infrared and visible light binocular cameras, and produces a smoke point sample data set of dual-light fusion images of infrared images and visible light images, and utilizes the characteristic that the dual-light fusion image can better express the advantages of image data features to supplement the features of smoke points; combines the smoking behavior detection method based on deep learning, and enriches the sample semantic features and positioning features through the FPN+PAN structure; uses image adaptive scaling and filling to reduce the computational cost, reduce the deployment hardware requirements, improve the model recognition accuracy and real-time performance, and effectively realize smoking recognition and detection in infrared scenes; forms a smoking behavior detection system that can be deployed in factories, workshops and other workplaces to perform effective and stable smoking behavior detection.

[0020] The infrared and visible light binocular camera described in step 1 is installed at a height of 2.5m from the ground indoors, and the activity range of the people in the sample is marked with a detection range of 2m horizontally from the camera.

[0021] The initial sample range of the video shooting in step 2 is within two meters of the horizontal distance of the camera, and the target is detected to be smoking accompanied by normal communication, body movements, and walking back and forth.

[0022] The image fusion processing operation described in step 3 has the following specific steps:

[0023] 3.1. Use the Scharr operator to extract the edge of the visible light sample and obtain the smoking behavior feature map under the visible light sample;

[0024] 3.2. The visible light feature image I is transformed into RGB With infrared image I R Fusion is performed to obtain a dual-light fusion image; the specific formula is: I = I R +ФI RGB ,The samples after image fusion not only preserve the characteristic values ​​of ,smoking behavior in the infrared image, but also add the characteristic ,values ​​of smoking behavior to the original infrared image through the characteristic ,image of visible light, making the final output sample more contrasting ,and more obvious contours, which is conducive to feature expression.

[0025] The mosaic enhancement of the input sample described in step 5 is to first splice the four samples by random arrangement, random scaling, and random cropping, so that the background of the smoke point sample is richer, the small target detection accuracy is improved, and in this way, the data input of four pictures is completed at one time during the calculation process, reducing the calculation cost.

[0026] Step 6: The FPN+PAN structure is used to process the backbone network to extract the smoke point features. The specific steps are as follows:

[0027] 6.1. First, the sample is extracted through the FPN structure to obtain feature maps of three sizes: 80×80, 40×40, and 20×20. The feature map of size 20×20 is upsampled to 40×40 and fused with the original medium-sized feature map. The fused feature map is then upsampled twice again and fused with the original large-sized feature map to obtain a feature map of 80×80 for prediction. From top to bottom, the high-level information is transferred and fused by upsampling.

[0028] 6.2. Then, the feature maps of three sizes extracted by the FPN structure are downsampled to 40×40 through PAN, and fused with the medium-sized feature map in the FPN structure. The fused feature map is then downsampled to 20×20 again and fused with the small-sized feature map in the FPN structure to convey the positioning features from the bottom to the top.

[0029] An infrared image smoking detection system based on deep learning, comprising:

[0030] An acquisition module, used for acquiring image data through a workplace monitoring device, and defining the image data as a source image for acquisition and storage;

[0031] A fusion module is used to perform image fusion processing operations on the collected visible light initial samples and infrared initial samples to obtain fused samples;

[0032] The dataset creation module is used to create preset training sets, test sets, and validation sets.

[0033] The smoke point feature extraction module is used to compare the features of the fused samples one by one using the preset training samples, determine the area of ​​the smoke point features, and extract the smoke point features;

[0034] The smoke point detection module uses non-maximum suppression (NMS) to filter the smoke point feature extraction frame, retain the prediction frame with the highest confidence, complete the smoke point detection process, and obtain the result of whether there is smoking behavior;

[0035] The smoking alarm module sends instructions to the alarm device according to the detection results of the smoke point detection module to complete the alarm.

[0036] The monitoring device is an infrared and visible light binocular camera.

[0037] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0038] The present invention uses image fusion technology to fuse visible light and infrared images, so that the fused image contains richer feature information, and improves the accuracy and reliability of the fused image;

[0039] The present invention uses Yolov5 as the detection network, improves the detection accuracy of small targets through Mosaic enhancement, and reduces the computational cost through adaptive image scaling to facilitate deployment;

[0040] The present invention combines the feature processing method of FPN+PAN, conveys semantic features from top to bottom, and conveys positioning features from bottom to top, and the two are combined to complete feature aggregation and improve detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0042] Figure 1 is an operation flow chart of the smoking detection system of the present invention;

[0043] Figure 2 is a flow chart of the smoke point detection algorithm of the present invention;

[0044] Figure 3 It is a structural diagram of the smoke point detection network model of the present invention;

[0045] Figure 4 This is a diagram showing the fusion effect of visible light and infrared images of the present invention;

[0046] Figure 5 This is a smoking detection effect diagram of the present invention; DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Example 1

[0050] like Figure 1-5 As shown, this embodiment provides an infrared image smoking detection method based on deep learning. The smoking behavior images are collected by infrared and visible light binocular cameras, and a smoke point sample data set of dual-light fusion images of infrared images and visible light images is produced. The dual-light fusion image can better express the advantages of image data features to supplement the features of smoke points; the idea of ​​deep learning is used to fully extract the semantic features of smoke points by training the network model, so as to solve the problems of unstable smoke point detection accuracy and poor positioning effect in infrared scenes. The specific steps are as follows:

[0051] Step 1: Install the hardware environment. First, according to the security camera installation standards, install the infrared and visible light binocular camera at a height of 2.5m from the ground indoors while meeting the requirements of the monitoring target field of view. Mark the range of human activities with a detection range of 2m horizontally from the camera.

[0052] Step 2: Data collection: Use a binocular camera to shoot a video of a simulated workplace, and make the target person smoke within a two-meter horizontal distance from the camera, accompanied by normal communication, body movements, walking back and forth, etc., to collect smoking behavior videos of three types of simulated real smoking scenes: single person, double person, and multiple person. At the same time, obtain visible light video and infrared video, and take frames at the same time starting point and the same frequency to obtain visible light initial samples and infrared initial samples;

[0053] Step 3: Data preprocessing. In order to integrate the information of the same smoking behavior captured by infrared images and visible light images at the same time, image fusion processing is performed on the visible light samples and infrared samples. The edge of the visible light samples is extracted using the Scharr operator to obtain the smoking behavior feature map under the visible light samples. Then, the visible light feature image I is weighted by the weight operator Ф. RGB With infrared image I R Fusion is performed to obtain a dual-light fusion image; the specific formula is: I = I R +ФI RGB ,The samples after image fusion not only preserve the characteristic values ​​of smoking behavior in the infrared image, but also add the characteristic ,values ​​of smoking behavior to the original infrared image through the characteristic image of visible light, making the final output sample more contrasting ,and more obvious contours, which is conducive to feature expression;

[0054] Step 4: Dataset creation, screening the fused samples, screening 2700 images and annotating them, the samples are dual-light fusion images with a size of 640×480×3. The 2700 annotated samples are divided into training set, test set and validation set in a ratio of 8:1:1;

[0055] Step 5: Smoke point feature extraction. The present invention adopts the Yolov5 network model, which is a lightweight model with fast detection speed and excellent detection accuracy. First, the input sample is Mosaic enhanced. The four samples are spliced ​​by random arrangement, random scaling, and random cropping to make the background of the smoke point sample richer and improve the accuracy of small target detection. In this way, the data input of four pictures is completed at one time during the calculation process, reducing the calculation cost. After the original sample is adaptively scaled and filled, the input sample size becomes 640×640×3, which is input into the backbone network. The sample is sliced ​​by the Focus structure to obtain 320×320×12, and then a feature map is obtained by convolution operation with 64 convolution kernels. The feature extraction is completed through the backbone network.

[0056] Step 6: Smoke point feature fusion, Yolov5 uses the structure of FPN (Feature Pyramid Network) + PAN (Perceptual Adversarial Network) to process the features extracted by the backbone network;

[0057] First, the sample is extracted through the FPN structure to obtain feature maps of three sizes: 80×80, 40×40, and 20×20. The feature map of size 20×20 is upsampled to 40×40 and fused with the original medium-sized feature map. The fused feature map is then upsampled twice again and fused with the original large-sized feature map to obtain a feature map of 80×80 for prediction. From top to bottom, the high-level information is transferred and fused by upsampling.

[0058] Then, the three-size feature maps extracted by FPN are downsampled to 40×40 through PAN, and fused with the medium-size feature map in the FPN structure. Then, the fused feature map is downsampled to 20×20 again and fused with the small-size feature map in the FPN structure to convey the positioning features from bottom to top.

[0059] Step 7: Detect smoke points. In the post-processing of smoke point detection, the target frame is screened through non-maximum suppression (NMS), and the prediction frame with the highest confidence is retained to complete the smoke point detection process. The image with smoke points is judged as someone smoking.

[0060] Step 8: Smoke point alarm. Deploy the smoke point detection model to the terminal processor. Transmit the video captured by the infrared and visible light binocular camera to the terminal processor via Ethernet. Implement real-time smoke point monitoring within the camera shooting range. Once a smoke point is detected, send a command to the alarm device via the serial port to complete the alarm.

[0061] The detection accuracy and detection frame rate of the Yolov5 detection method of the present invention are significantly better than those of the Yolov1 and FasterRCNN detection methods. The comparison table of smoking detection methods is as follows:

[0062] Detection Methods Detection accuracy Detection frame rate (fps) Yolov1 67.95％ 43 Faster RCNN 78.43％ 76 Yolov5 83.56％ 96

Claims

1. A method for detecting smoking in infrared images based on deep learning, characterized in that: The steps include: Step 1: Install the hardware environment and install the infrared and visible light binocular cameras according to the security camera installation standards, and electrically connect them to the terminal processor respectively; Step 2: Data collection: Use an infrared visible binocular camera to shoot a video of a simulated workplace, collect smoking behavior videos of three types of simulated real smoking scenes: single person, double person, and multiple person, and obtain visible light video and infrared video at the same time, and take frames at the same time starting point and frequency to obtain visible light initial samples and infrared initial samples; Step 3: Data preprocessing: performing image fusion processing on the visible light initial sample and the infrared initial sample to obtain a fused sample; Step 4: Dataset creation: Filter the fused samples to obtain 2,700 images and annotate them. The samples are dual-light fusion images with a size of 640×480×3. The 2,700 annotated samples are divided into training set, test set and validation set in a ratio of 8:1:

1. Step 5: Extract smoke point features. Use the Yolov5 network model to perform Mosaic enhancement on the input samples. After the original samples are adaptively scaled and filled, the input sample size becomes 640×640×3. Input it into the backbone network, slice the sample into 320×320×12 through the Focus structure, and then perform a convolution operation with 64 convolution kernels to obtain a feature map. The backbone network completes the smoke point feature extraction. Step 6: Fusion of smoke point features. Yolov5 uses the FPN+PAN structure to process the backbone network to extract the smoke point features. Step 7: Detect smoke points. Use non-maximum suppression (NMS) to filter the target frame and retain the prediction frame with the highest confidence. This completes the smoke point detection process and determines that images with smoke points are smoked. Step 8: Smoke point alarm: deploy the smoke point detection model to the terminal processor, transmit the video captured by the infrared and visible light binocular camera to the terminal processor via Ethernet, implement real-time smoke point monitoring in the camera shooting range, and once a smoke point is detected, send a command to the alarm device via the serial port to complete the alarm; The initial sample range of the video captured in step 2 is within two meters of the horizontal distance of the camera, and the target is detected to be smoking accompanied by normal communication, body movements, and walking back and forth; The image fusion processing operation described in step 3 has the following specific steps: 3.

1. Use the Scharr operator to extract the edge of the visible light sample and obtain the smoking behavior feature map under the visible light sample; 3.

2. The visible light feature image IRGB and the infrared image IR are fused by the weight operator Ф to obtain a dual-light fusion image; the specific formula is: I = IR + ФIRGB; Step 6: The FPN+PAN structure is used to process the backbone network to extract the smoke point features. The specific steps are as follows: 6.

1. First, the sample is extracted through the FPN structure to obtain feature maps of three sizes: 80×80, 40×40, and 20×20. The feature map of size 20×20 is upsampled to 40×40 and fused with the original medium-sized feature map. The fused feature map is then upsampled twice again and fused with the original large-sized feature map to obtain a feature map of 80×80 for prediction. From top to bottom, the high-level information is transferred and fused by upsampling. 6.

2. Then the three-size feature maps extracted by FPN are passed through PAN, and the fused feature map with a size of 80×80 is downsampled to 40×40, and fused with the medium-sized feature map in the FPN structure. The fused feature map is then downsampled to 20×20 again and fused with the small-size feature map in the FPN structure to convey the positioning features from bottom to top.

2. The infrared image smoking detection method based on deep learning according to claim 1 is characterized in that: The infrared and visible light binocular camera described in step 1 is installed at a height of 2.5m from the ground indoors, and the activity range of the people in the sample is marked with a detection range of 2m horizontally from the camera.

3. The infrared image smoking detection method based on deep learning according to claim 1 is characterized in that: In step 5, the Mosaic enhancement of the input sample is performed by first splicing the four samples by random arrangement, random scaling, and random cropping.

4. A deep learning based infrared image smoking detection system using the deep learning based infrared image smoking detection method as described in any one of claims 1 to 3, characterized in that: include: An acquisition module, used for acquiring image data through a workplace monitoring device, and defining the image data as a source image for acquisition and storage; A fusion module is used to perform image fusion processing operations on the collected visible light initial samples and infrared initial samples to obtain fused samples; The smoke point feature extraction module is used to compare the features of the fused samples one by one using the preset training samples, determine the area of ​​the smoke point features, and extract the smoke point features; The smoke point detection module uses non-maximum suppression (NMS) to filter the smoke point feature extraction frame, retain the prediction frame with the highest confidence, complete the smoke point detection process, and obtain the result of whether there is smoking behavior; The smoking alarm module sends instructions to the alarm device to complete the alarm according to the detection results of the smoke point detection module.

5. The deep learning-based infrared image smoking detection system according to claim 4, characterized in that: It also includes a dataset creation module for creating preset training sets, test sets, and validation sets.

6. The deep learning-based infrared image smoking detection system according to claim 4, characterized in that: The monitoring device is an infrared and visible light binocular camera.

Images