A target tracking smoking detection method and device based on deep learning
Patent Information
- Application Number
- CN202611381819.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-08
- Publication Date
- 2026-10-13
AI Technical Summary
[0005]有鉴于此，本发明的目的在于提供一种基于深度学习的目标追踪吸烟检测方法及装置，旨在解决现有技术中，检测模型效率低、检测精度不足、缺乏追踪与反馈机制的问题
本申请在现有的YOLOv8n目标检测模型的基础上，将主干网络中的原C2f模块替换为C2f-DC模块，C2f-DC模块通过深度可分离卷积显著降低了模型参数量和计算量，同时提升了特征提取能力；通过在原特征融合网络之后引入SCP模块，增强了模型对全局上下文信息的理解，提高了对吸烟小目标的检测精度；训练数据集包含可见光与红外双模态，使模型能适应白天、黑夜、不同天气和场景，具有极强的鲁棒性；将改进的检测模型与ByteTrack追踪算法结合，实现了对吸烟人员的实时定位与持续追踪，并联动报警装置进行精确劝导，形成了“检测-追踪-反馈”的智能闭环系统，有效提升了禁烟管理效率。
Smart Images
Figure CN122906757A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and intelligent monitoring technology, specifically to a target tracking and smoking detection method and device based on deep learning. Background Technology
[0002] Banning smoking in public places is an important measure to protect public health. However, traditional smoking ban monitoring relies on manual review of video footage, which is inefficient and has blind spots.
[0003] Existing video-based smoking detection methods typically suffer from the following problems:
[0004] The detection model has a large number of parameters, making it difficult to deploy in a real-time monitoring system; It has low detection accuracy in complex lighting conditions (such as at night) and for small targets (such as cigarette butts); It can only identify smoking actions, but it cannot continuously track and persuade smokers, and lacks a complete closed-loop processing capability. Summary of the Invention
[0005] In view of this, the purpose of the present invention is to provide a target tracking and smoking detection method and apparatus based on deep learning, which aims to solve the problems of low detection model efficiency, insufficient detection accuracy, and lack of tracking and feedback mechanisms in the prior art.
[0006] According to a first aspect of the present invention, a deep learning-based target tracking and smoking detection method is provided, the method comprising: By collecting pedestrian smoking datasets through surveillance cameras, and combining the collected pedestrian smoking datasets with public datasets, pedestrian detection and smoking detection datasets are created. The pre-built model framework is iteratively trained using the pedestrian detection and smoking detection datasets to obtain a target smoking detection model. The target smoking detection model includes a backbone network, a feature aggregation network, a spatial context pyramid (SCP) module, and a detection head. The output of the backbone network is connected to the input of the feature aggregation network, the output of the feature aggregation network is connected to the input of the spatial context pyramid (SCP) module, and the output of the spatial context pyramid (SCP) module is connected to the detection head. The detection head outputs the smoking detection result. The backbone network includes a C2f-DC module. The convolutional module DCBS within the C2f-DC module uses depthwise separable convolution to reduce the number of parameters and computational cost, and introduces a convolutional block attention module CBAM to simultaneously focus on features in both channel and spatial dimensions. The spatial context pyramid (SCP) module uses a context aggregation block CABlock with residual connections to fuse local and global information. Finally, the detection head outputs the detection result. Deploy a target smoking detection model, use the ByteTrack target tracking algorithm to track all personnel in the monitoring area in real time, and use the target smoking detection model to detect smoking behavior of the personnel tracked in real time in the monitoring area; When smoking is detected in the monitored area, the smoking behavior is prevented through the reminder device in the monitored area.
[0007] Preferably, The process of collecting pedestrian smoking datasets via surveillance cameras, and then creating pedestrian detection and smoking detection datasets based on these datasets and in conjunction with public datasets, includes: An initial pedestrian smoking dataset is collected around the clock using a surveillance camera with infrared night vision capabilities. The initial pedestrian smoking dataset includes two modalities: daytime visible light color images and nighttime infrared images. An enhanced pedestrian smoking dataset is obtained by augmenting the initial pedestrian smoking dataset through geometric transformation and brightness transformation. The geometric transformation includes performing at least one of translation, rotation, stitching, and scaling on the initial pedestrian smoking dataset. The brightness transformation includes performing at least one of HSV color adjustment and contrast transformation on the initial pedestrian smoking dataset. The pedestrian detection and smoking detection datasets are obtained by combining the enhanced pedestrian smoking dataset and the public dataset. The pedestrian detection and smoking detection datasets contain data under different lighting conditions, different scenes, different weather conditions, and different modalities.
[0008] Preferably, The C2f-DC module consists of a convolutional module DCBS and multiple bottleneck modules connected by residuals. Input features are fed into the convolutional module DCBS for depthwise separable convolution. Channel separation is performed on the output features of the convolutional module DCBS. The channel-separated features are fed into the first bottleneck module (Bottleneck) to extract residual features and then directly into the channel fusion module. The output features of each bottleneck module (Bottleneck) are fed into the next bottleneck module (Bottleneck) to continue extracting residual features and then directly into the channel fusion module. All features entering the channel fusion module are fused to obtain the output features of the C2f-DC module. The convolutional module DCBS performs depthwise separable convolution on the input features, then performs batch normalization, and activates the batch-normalized features through the SiLU activation function to obtain the output features of the convolutional module DCBS. The depthwise separable convolution operation includes two concatenated operations: depthwise convolution and pointwise convolution. Depthwise convolution performs convolution on each input channel independently, while pointwise convolution uses a 1×1 convolution to fuse the channels of the output of the depthwise convolution. The bottleneck module Bottleneck consists of two convolutional modules DCBS and one convolutional block attention module CBAM connected in series. The input features first enter the first convolutional module DCBS to extract convolutional features. The output of the first convolutional module DCBS enters the second convolutional module DCBS to extract convolutional features. The output of the second convolutional module DCBS enters the convolutional block attention module CBAM to extract fused features of spatial attention and channel attention. The input features of the bottleneck module Bottleneck and the output features of the convolutional block attention module CBAM are added together to obtain the output features of the bottleneck module Bottleneck.
[0009] Preferably, The Convolutional Block Attention (CBAM) module adopts a serial structure, first passing through the channel attention module and then the spatial attention module. The channel attention module performs global average pooling and global max pooling operations on the input features, feeds the two pooling results into a shared multilayer perceptron, adds them together, and then activates them with a sigmoid function to generate channel weights. The channel weights are multiplied by the input features to obtain the input features of the spatial attention module. The spatial attention module performs average pooling and max pooling operations on the input features along the channel dimension, concatenates the two pooling results along the channel dimension, and generates spatial weights through a 7×7 convolution and sigmoid activation. The spatial weights are multiplied by the input features of the spatial attention module to obtain the output features of the CBAM module.
[0010] Preferably, The SCP module employs a context aggregation block CABlock with residual connections to fuse local and global information, including: The SCP module consists of a context aggregation block CABlock. The input features of the SCP module are entered into the context aggregation block CABlock for feature extraction. The output features of the context aggregation block CABlock are added to the input features of the SCP module to obtain the output features of the SCP module.
[0011] Preferably, The context aggregation block CABlock is based on the self-attention mechanism of Transformer. It obtains the position output of each pixel by adding the original position input of each pixel to a weighted aggregated global context. The position outputs of all pixels are aggregated to obtain the output features of the context aggregation block CABlock. The weights of the weighted aggregation are obtained by linear transformation and Softmax normalization of the features at the pixel location.
[0012] Preferably, The target smoking detection model also includes: The Spatial Pyramid Pooling (SPP) module in the backbone network is replaced with the Fast Spatial Pyramid Pooling (SPPF) module. The Fast Spatial Pyramid Pooling (SPPF) module uses serial small convolutional kernels to achieve the receptive field of large convolutional kernels, thereby expanding the receptive field while reducing the number of parameters.
[0013] Preferably, The method of preventing smoking through a reminder device within the monitored area includes: When the same person is detected smoking within Y consecutive frames of images, the smoking signal and the person's location information are sent to the audible and visual alarm device and the mobile robot in the corresponding monitoring area, respectively. The audible and visual alarm device will issue a warning, and the mobile robot will automatically navigate to the person's location and play a voice admonition.
[0014] According to a second aspect of the present invention, a deep learning-based target tracking and smoking detection device is provided, the device comprising: Dataset creation module: Used to collect pedestrian smoking datasets through surveillance cameras, and based on the collected pedestrian smoking datasets and combined with public datasets, to create pedestrian detection and smoking detection datasets; Training Module: Used to iteratively train a pre-built model framework using the pedestrian detection and smoking detection datasets to obtain a target smoking detection model. The target smoking detection model includes a backbone network, a feature aggregation network, a spatial context pyramid (SCP) module, and a detection head. The output of the backbone network is connected to the input of the feature aggregation network, the output of the feature aggregation network is connected to the input of the spatial context pyramid (SCP) module, and the output of the spatial context pyramid (SCP) module is connected to the detection head. The detection head outputs the smoking detection result. The backbone network includes a C2f-DC module. The convolutional module DCBS within the C2f-DC module uses depthwise separable convolution to reduce the number of parameters and computational cost, and introduces a convolutional block attention module CBAM to simultaneously focus on features in both channel and spatial dimensions. The spatial context pyramid (SCP) module uses a context aggregation block CABlock with residual connections to fuse local and global information. Finally, the detection head outputs the detection result. Deployment module: Used to deploy the target smoking detection model, track all personnel in the monitoring area in real time using the ByteTrack target tracking algorithm, and detect smoking behavior of the personnel tracked in real time in the monitoring area using the target smoking detection model; Alert module: Used to prevent smoking by using an alert device within the monitored area when smoking is detected.
[0015] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This application, based on the existing YOLOv8n target detection model, replaces the original C2f module in the backbone network with a C2f-DC module. The C2f-DC module significantly reduces the number of model parameters and computational cost through depthwise separable convolution, while improving feature extraction capabilities. By introducing an SCP module after the original feature fusion network, the model's understanding of global contextual information is enhanced, improving the detection accuracy of small smoking targets. The training dataset includes both visible light and infrared modes, enabling the model to adapt to day and night, different weather conditions and scenarios, and exhibiting strong robustness. By combining the improved detection model with the ByteTrack tracking algorithm, real-time location and continuous tracking of smokers are achieved, and alarm devices are linked for precise persuasion, forming an intelligent closed-loop system of "detection-tracking-feedback," effectively improving the efficiency of smoking ban management.
[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description
[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0018] Figure 1 This is a flowchart illustrating a deep learning-based target tracking and smoking detection method according to an exemplary embodiment; Figure 2 This is a schematic diagram of the overall framework of an improved YOLOv8n target detection model according to another exemplary embodiment; Figure 3 This is a schematic diagram of the C2f-DC module according to another exemplary embodiment; Figure 4 This is a schematic diagram of the C2f module according to another exemplary embodiment; Figure 5 This is a schematic diagram of the framework of a Convolutional Block Attention (CBAM) module according to another exemplary embodiment; Figure 6 This is a schematic diagram of the framework of an SCP module according to another exemplary embodiment; Figure 7 This is a schematic diagram of the framework of a fast spatial pyramid pooling SPPF module according to another exemplary embodiment; Figure 8 This is a schematic diagram of device deployment according to another exemplary embodiment; Figure 9 This is a schematic diagram of a deep learning-based target tracking and smoking detection device according to another exemplary embodiment; In the attached diagram: 1-Dataset creation module, 2-Training module, 3-Deployment module, 4-Notification module. Detailed Implementation
[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.
[0020] Example 1: Figure 1 This is a flowchart illustrating a deep learning-based target tracking and smoking detection method according to an exemplary embodiment, as shown below. Figure 1 As shown, the method includes: S1. Collect pedestrian smoking datasets through surveillance cameras. Based on the collected pedestrian smoking datasets and combined with public datasets, create pedestrian detection and smoking detection datasets. S2, the pre-built model framework is iteratively trained using the pedestrian detection and smoking detection datasets to obtain a target smoking detection model. The target smoking detection model includes a backbone network, a feature aggregation network, a spatial context pyramid (SCP) module, and a detection head. The output of the backbone network is connected to the input of the feature aggregation network, the output of the feature aggregation network is connected to the input of the spatial context pyramid (SCP) module, and the output of the spatial context pyramid (SCP) module is connected to the detection head. The detection head outputs the smoking detection result. The backbone network includes a C2f-DC module. The convolutional module DCBS within the C2f-DC module uses depthwise separable convolution to reduce the number of parameters and computational cost, and introduces a convolutional block attention module CBAM to simultaneously focus on features in both channel and spatial dimensions. The spatial context pyramid (SCP) module uses a context aggregation block CABlock with residual connections to fuse local and global information. Finally, the detection head outputs the detection result. S3, Deploy a target smoking detection model, use the ByteTrack target tracking algorithm to track all personnel in the monitoring area in real time, and use the target smoking detection model to detect smoking behavior of the personnel tracked in real time in the monitoring area; S4. When smoking is detected in the monitored area, the smoking behavior is prevented through the reminder device in the monitored area. It is understood that the target smoking detection model in this application is an improvement on the existing YOLOv8n target detection model. The existing YOLOv8n target detection model consists of three main parts: the backbone network, the feature aggregation network, and the detection head. The improvements to the existing YOLOv8n target detection model include: replacing the C2f module in the original backbone network with the C2f-DC module, and replacing the Spatial Pyramid Pooling (SPP) module in the original backbone network with the Fast Spatial Pyramid Pooling (SPPF) module; the feature aggregation network remains unchanged, and a Spatial Context Pyramid (SCP) module is added after the feature aggregation network; the improved YOLOv8n target detection model is attached. Figure 2 As shown; The structure of the C2f-DC module in the improved YOLOv8n object detection model is shown in the attached figure. Figure 3 As shown in the attached diagram, the C2f module of the original YOLOv8n object detection model... Figure 4 As shown, the difference is that all convolutions in the C2f-DC module are depthwise separable convolutions, and at the same time, the convolutional block attention module CBAM, which is not present in the original bottleneck module Bottleneck of the C2f module, is introduced. It pays attention to information in both channel and spatial dimensions at the same time, effectively improving the feature learning ability of the backbone network. The C2f-DC module consists of a convolutional module DCBS and multiple bottleneck modules connected by residuals. The input features are fed into the convolutional module DCBS for depthwise separable convolution. The output features of the convolutional module DCBS are subjected to channel separation. The channel-separated features are fed into the first bottleneck module (Bottleneck) to extract residual features, and then directly into the channel fusion module. The output features of each bottleneck module (Bottleneck) are fed into the next bottleneck module (Bottleneck) to extract residual features, and then directly into the channel fusion module. All features entering the channel fusion module are fused to obtain the output features of the C2f-DC module. After performing depthwise separable convolution on the input features, the convolution module DCBS performs batch normalization. The batch-normalized features are then activated by the SiLU activation function to obtain the output features of the convolution module DCBS. Depthwise separable convolution uses depthwise convolution and pointwise convolution to perform convolution operations. Compared with standard convolution (the original C2f module), this module has significantly reduced parameters and computational cost. For example, the input feature size is D F × D FThe input channel and output channel are respectively M and N The kernel size is D K × D K The number of parameters and computational cost of standard convolution are as follows:
[0021]
[0022] In the formula, The number of parameters representing a standard convolution. This represents the total number of floating-point operations in standard convolution; The number of parameters in a depthwise separable convolution is equal to the sum of the parameters in a depthwise convolution and a pointwise convolution, as follows:
[0023]
[0024] In the formula, The number of parameters representing a depthwise separable convolution. This represents the total number of floating-point operations performed in depth-separable convolution. The parameter count and computational cost of depthwise separable convolution and standard convolution are as follows:
[0025]
[0026] Under normal circumstances, N Very large, 1 / N If we disregard these factors, then depthwise separable convolution reduces both the number of parameters and computational cost compared to standard convolution to the original level. D K One-squared of the original size, the commonly used convolution kernel size is 3×3. In this way, the number of parameters and the amount of computation are reduced to 1 / 9 of the original, effectively reducing the complexity of the model. The bottleneck module Bottleneck consists of two convolutional modules DCBS and one convolutional block attention module CBAM connected in series. The input features first enter the first convolutional module DCBS to extract convolutional features. The output of the first convolutional module DCBS enters the second convolutional module DCBS to extract convolutional features. The output of the second convolutional module DCBS enters the convolutional block attention module CBAM to extract fused features of spatial attention and channel attention. The input features of the bottleneck module Bottleneck and the output features of the convolutional block attention module CBAM are added together to obtain the output features of the bottleneck module Bottleneck. As attached Figure 5As shown, the Convolutional Block Attention (CBAM) module uses a serial structure to combine the channel attention module and the spatial attention module. The channel attention module is prioritized over the spatial attention module to achieve better results. The specific calculation process is as follows:
[0027]
[0028]
[0029]
[0030] In the formula, F As input features, Indicates channel attention output, Indicates spatial attention output; The intermediate feature represents the pointwise product of the channel attention output and the input feature; F For output features, MLP For multilayer perceptrons, AvgPool and MaxPool For average and maximum pooling operations, This indicates a convolution operation with a 7×7 kernel; Through this mechanism, the network can simultaneously focus on the key channel characteristics (such as the heat source of the cigarette butt) and key spatial location of the "smoking" behavior; As attached Figure 6 As shown, the original YOLOv8n feature aggregation network is used to fuse feature maps from different levels of the backbone network. This application introduces the Spatial Context Pyramid (SCP) module after it. The SCP module is a residual module with CABlock (contextAggregation block), which effectively fuses local and global information while reducing information clutter, thereby enhancing the model's ability to understand the surrounding environment of the target. The CABlock module has a structure similar to Transformer, and its specific expression is shown below:
[0031] In the formula, and For input, For output, and The index for each pixel, and It is a linear transformation matrix; It is a reweighted matrix used to balance the degree of global aggregation for each pixel, and its specific expression is shown below:
[0032] In the formula, and For input, and For the matrix index, It is a linear transformation matrix; The meaning of the above formula is as follows: Step 1: First, for each pixel position j Using a learnable linear transformation matrix Its eigenvectors Perform a linear mapping to obtain the key at that location. Then, apply this key to all pixels in the local region or the entire feature map containing that pixel. N i pixel position (index is) m ), calculate the mapped exponent values respectively. And using the sum of these index values as the denominator, As a molecule, the attention weight of this pixel relative to all pixels is obtained through Softmax normalization. The physical meaning of this weight is: pixel j Features and each surrounding pixel m The similarity between features; Step 2: Then use another linear transformation matrix For pixels j Features Perform mapping to obtain the value at that location, then multiply each attention weight obtained in the first step by the value at the corresponding location, and apply this to all... N i The summation is performed at each position to obtain the global context aggregation result for that pixel; Step 3: To adaptively adjust the degree to which each pixel participates in global aggregation, a reweighted scalar is computed. The coefficient is calculated using the third linear transformation matrix. For pixels j Features Perform mapping, then Softmax normalization, i.e. equal Divide by all pixel positions (index) n )of The sum of these factors determines the extent to which the global context information of the current pixel should be incorporated into the original features. Step 4: Finally, reweight the coefficients obtained in step 3. Multiply by the weighted aggregate value obtained in the second step, and then multiply by the pixel value. jOriginal input features Add them together to get the final output at that position. ; The entire CABlock achieves adaptive fusion of the "original information of each pixel location" and the "information obtained by aggregation based on the global context". In addition, the SCP module sets up residual connections outside the CABlock to add the output of the CABlock to the input of the entire SCP module, further alleviating the information loss problem in deep networks and enhancing gradient flow, so that the model can learn the complex relationship between smoking behavior and the surrounding environment (such as hand posture, facial region, cigarette butt halo, etc.) more effectively. As attached Figure 7 As shown, in the backbone network of the improved YOLOv8n object detection model, the original SPP is replaced by the Fast Feature Pyramid Pooling (SPPF) module to expand the receptive field. Compared with the original SPP, SPPF uses a serial structure instead of the original parallel structure and uses small kernel convolution to expand the receptive field. The specific principle is that a large kernel convolution operation is equivalent to two small kernel convolution operations. For example, a 5×5 convolution operation is equivalent to two 3×3 convolution operations. SPPF replaces the original SPP module and achieves the same effect with fewer parameters. After introducing the improved YOLOv8n object detection model, the specific implementation steps of this application are explained as follows: Data Acquisition and Composition: Using surveillance cameras equipped with infrared night vision, smoking behavior data is collected around the clock (24 hours). The data includes two modalities: visible light color images during the day and infrared grayscale images at night. At the same time, combined with public smoking datasets, a rich dataset is finally formed that includes different lighting conditions (strong light, weak light, backlight), different scenes (indoor, outdoor, corridor), different weather conditions (sunny, rainy, foggy), and different modalities (RGB, infrared). Data augmentation: To improve the model's generalization ability, the collected image data is augmented. Augmentation methods are divided into two categories: one is geometric transformation, including random translation, random rotation, image stitching (such as the Mosaic method), and random scaling; the other is photometric transformation, including adjusting the hue and saturation of the HSV color space, and adjusting the contrast and brightness. Model training: The improved YOLOv8n object detection model is iteratively trained using data-augmented pedestrian detection and smoking detection datasets until a preset number of iterations is reached (usually 100 to 200 rounds), or after 20 to 50 consecutive iterations, the model's loss value decreases below a preset threshold, thus satisfying the stopping condition and obtaining the target smoking detection model. Model Deployment: Deploy the trained target smoking detection model and combine it with the ByteTrack target tracking algorithm to perform real-time ID assignment and trajectory tracking for all personnel in the monitoring screen; Smoking detection: In the same detection process, the target smoking detection model is used to identify smoking behavior (such as detecting cigarettes, smoke, etc.) within the target area of the tracked person. Finally, the structured information such as ID, location, and smoking status (yes / no) of each target is output. Smoking prevention: When smoking is detected, the system immediately triggers the sound and light alarm devices in the area where the smoker is located to issue a warning. At the same time, the smoker's precise location is sent to a mobile robot, which automatically navigates to the vicinity of the target and plays a voice message (such as "Hello, this is a no-smoking area, please do not smoke") to achieve precise and timely intervention.
[0033] This embodiment also provides a schematic diagram of the device deployment of this application, as shown in the attached diagram. Figure 8 As shown, a real-time smoking tracking and detection algorithm (target smoking detection model) is deployed to the server, all monitoring cameras are connected to the server and visualization platform through switches, and wireless communication between the mobile robot and the server is realized. The server processes the video data captured by the camera in real time and transmits the detection results to the visualization platform via a switch. The smoking behavior signal and the location information of the smoker are transmitted to the sound and light alarm device. In this embodiment, a loudspeaker and a mobile robot are used to provide alarm reminders and robot verbal persuasion to discourage smoking behavior.
[0034] Example 2: Figure 9 This is a schematic diagram of a deep learning-based target tracking and smoking detection device according to another exemplary embodiment, the device comprising: Dataset creation module 1: Used to collect pedestrian smoking datasets through surveillance cameras, and based on the collected pedestrian smoking datasets and combined with public datasets, to create pedestrian detection and smoking detection datasets; Training Module 2: Used to iteratively train a pre-built model framework using the pedestrian detection and smoking detection datasets to obtain a target smoking detection model. The target smoking detection model includes a backbone network, a feature aggregation network, a spatial context pyramid (SCP) module, and a detection head. The output of the backbone network is connected to the input of the feature aggregation network, the output of the feature aggregation network is connected to the input of the spatial context pyramid (SCP) module, and the output of the spatial context pyramid (SCP) module is connected to the detection head. The detection head outputs the smoking detection result. The backbone network includes a C2f-DC module. The convolutional module DCBS within the C2f-DC module uses depthwise separable convolution to reduce the number of parameters and computational cost, and introduces a convolutional block attention module CBAM to simultaneously focus on features in both channel and spatial dimensions. The spatial context pyramid (SCP) module uses a context aggregation block CABlock with residual connections to fuse local and global information. Finally, the detection head outputs the detection result. Deployment Module 3: Used to deploy the target smoking detection model, which tracks all personnel in the monitoring area in real time using the ByteTrack target tracking algorithm, and detects smoking behavior of the personnel tracked in real time in the monitoring area using the target smoking detection model; Alert Module 4: Used to prevent smoking by using an alert device within the monitored area when smoking is detected.
[0035] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.
[0036] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.
[0037] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.
[0038] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0039] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0040] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0041] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.
[0042] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0043] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A deep learning-based target tracking and smoking detection method, characterized in that, The method includes: By collecting pedestrian smoking datasets through surveillance cameras, and combining the collected pedestrian smoking datasets with public datasets, pedestrian detection and smoking detection datasets are created. The pre-built model framework is iteratively trained using the pedestrian detection and smoking detection datasets to obtain a target smoking detection model. The target smoking detection model includes a backbone network, a feature aggregation network, a spatial context pyramid (SCP) module, and a detection head. The output of the backbone network is connected to the input of the feature aggregation network, the output of the feature aggregation network is connected to the input of the spatial context pyramid (SCP) module, and the output of the spatial context pyramid (SCP) module is connected to the detection head. The detection head outputs the smoking detection result. The backbone network includes a C2f-DC module. The convolutional module DCBS within the C2f-DC module uses depthwise separable convolution to reduce the number of parameters and computational cost, and introduces a convolutional block attention module CBAM to simultaneously focus on features in both channel and spatial dimensions. The spatial context pyramid (SCP) module uses a context aggregation block CABlock with residual connections to fuse local and global information. Finally, the detection head outputs the detection result. Deploy a target smoking detection model, use the ByteTrack target tracking algorithm to track all personnel in the monitoring area in real time, and use the target smoking detection model to detect smoking behavior of the personnel tracked in real time in the monitoring area; When smoking is detected in the monitored area, the smoking behavior is prevented through the reminder device in the monitored area.
2. The method according to claim 1, characterized in that, The process of collecting pedestrian smoking datasets via surveillance cameras, and then creating pedestrian detection and smoking detection datasets based on these datasets and in conjunction with public datasets, includes: An initial pedestrian smoking dataset is collected around the clock using a surveillance camera with infrared night vision capabilities. The initial pedestrian smoking dataset includes two modalities: daytime visible light color images and nighttime infrared images. An enhanced pedestrian smoking dataset is obtained by augmenting the initial pedestrian smoking dataset through geometric transformation and brightness transformation. The geometric transformation includes performing at least one of translation, rotation, stitching, and scaling on the initial pedestrian smoking dataset. The brightness transformation includes performing at least one of HSV color adjustment and contrast transformation on the initial pedestrian smoking dataset. The pedestrian detection and smoking detection datasets are obtained by combining the enhanced pedestrian smoking dataset and the public dataset. The pedestrian detection and smoking detection datasets contain data under different lighting conditions, different scenes, different weather conditions, and different modalities.
3. The method according to claim 1, characterized in that, The C2f-DC module consists of a convolutional module DCBS and multiple bottleneck modules connected by residuals. Input features are fed into the convolutional module DCBS for depthwise separable convolution. Channel separation is performed on the output features of the convolutional module DCBS. The channel-separated features are fed into the first bottleneck module (Bottleneck) to extract residual features and then directly into the channel fusion module. The output features of each bottleneck module (Bottleneck) are fed into the next bottleneck module (Bottleneck) to continue extracting residual features and then directly into the channel fusion module. All features entering the channel fusion module are fused to obtain the output features of the C2f-DC module. The convolutional module DCBS performs depthwise separable convolution on the input features, then performs batch normalization, and activates the batch-normalized features through the SiLU activation function to obtain the output features of the convolutional module DCBS. The depthwise separable convolution operation includes two concatenated operations: depthwise convolution and pointwise convolution. Depthwise convolution performs convolution on each input channel independently, while pointwise convolution uses a 1×1 convolution to fuse the channels of the output of the depthwise convolution. The bottleneck module Bottleneck consists of two convolutional modules DCBS and one convolutional block attention module CBAM connected in series. The input features first enter the first convolutional module DCBS to extract convolutional features. The output of the first convolutional module DCBS enters the second convolutional module DCBS to extract convolutional features. The output of the second convolutional module DCBS enters the convolutional block attention module CBAM to extract fused features of spatial attention and channel attention. The input features of the bottleneck module Bottleneck and the output features of the convolutional block attention module CBAM are added together to obtain the output features of the bottleneck module Bottleneck.
4. The method according to claim 3, characterized in that, The Convolutional Block Attention (CBAM) module adopts a serial structure, first passing through the channel attention module and then the spatial attention module. The channel attention module performs global average pooling and global max pooling operations on the input features, feeds the two pooling results into a shared multilayer perceptron, adds them together, and then activates them with a sigmoid function to generate channel weights. The channel weights are multiplied by the input features to obtain the input features of the spatial attention module. The spatial attention module performs average pooling and max pooling operations on the input features along the channel dimension, concatenates the two pooling results along the channel dimension, and generates spatial weights through a 7×7 convolution and sigmoid activation. The spatial weights are multiplied by the input features of the spatial attention module to obtain the output features of the CBAM module.
5. The method according to claim 1, characterized in that, The SCP module employs a context aggregation block CABlock with residual connections to fuse local and global information, including: The SCP module consists of a context aggregation block CABlock. The input features of the SCP module are entered into the context aggregation block CABlock for feature extraction. The output features of the context aggregation block CABlock are added to the input features of the SCP module to obtain the output features of the SCP module.
6. The method according to claim 5, characterized in that, The context aggregation block CABlock is based on the self-attention mechanism of Transformer. It obtains the position output of each pixel by adding the original position input of each pixel to a weighted aggregated global context. The position outputs of all pixels are aggregated to obtain the output features of the context aggregation block CABlock. The weights of the weighted aggregation are obtained by linear transformation and Softmax normalization of the features at the pixel location.
7. The method according to claim 1, characterized in that, The target smoking detection model also includes: The Spatial Pyramid Pooling (SPP) module in the backbone network is replaced with the Fast Spatial Pyramid Pooling (SPPF) module. The Fast Spatial Pyramid Pooling (SPPF) module uses serial small convolutional kernels to achieve the receptive field of large convolutional kernels, thereby expanding the receptive field while reducing the number of parameters.
8. The method according to claim 1, characterized in that, The method of preventing smoking through a reminder device within the monitored area includes: When the same person is detected smoking within Y consecutive frames of images, the smoking signal and the person's location information are sent to the audible and visual alarm device and the mobile robot in the corresponding monitoring area, respectively. The audible and visual alarm device will issue a warning, and the mobile robot will automatically navigate to the person's location and play a voice admonition.
9. A target tracking and smoking detection device based on deep learning, characterized in that, The device includes: Dataset creation module: Used to collect pedestrian smoking datasets through surveillance cameras, and based on the collected pedestrian smoking datasets and combined with public datasets, to create pedestrian detection and smoking detection datasets; Training Module: Used to iteratively train a pre-built model framework using the pedestrian detection and smoking detection datasets to obtain a target smoking detection model. The target smoking detection model includes a backbone network, a feature aggregation network, a spatial context pyramid (SCP) module, and a detection head. The output of the backbone network is connected to the input of the feature aggregation network, the output of the feature aggregation network is connected to the input of the spatial context pyramid (SCP) module, and the output of the spatial context pyramid (SCP) module is connected to the detection head. The detection head outputs the smoking detection result. The backbone network includes a C2f-DC module. The convolutional module DCBS within the C2f-DC module uses depthwise separable convolution to reduce the number of parameters and computational cost, and introduces a convolutional block attention module CBAM to simultaneously focus on features in both channel and spatial dimensions. The spatial context pyramid (SCP) module uses a context aggregation block CABlock with residual connections to fuse local and global information. Finally, the detection head outputs the detection result. Deployment module: Used to deploy the target smoking detection model, track all personnel in the monitoring area in real time using the ByteTrack target tracking algorithm, and detect smoking behavior of the personnel tracked in real time in the monitoring area using the target smoking detection model; Alert module: Used to prevent smoking by using an alert device within the monitored area when smoking is detected.