Smoking detection method and device and electronic equipment

Abstract

The invention provides a smoking detection method and device and electronic equipment, and belongs to the technical field of deep learning, and the method comprises the steps: inputting a video image into a human body detection model, and obtaining a human body region image; inputting the human body region image into a multi-target tracking algorithm model to obtain continuous frames of the target person; inputting the continuous frames of the target person into a behavior recognition model to obtain image frames of the smoking action; inputting the image frame of the smoking action into a cigarette target detection model to obtain a detection result of whether the target person smokes or not; the human body detection model and the cigarette target detection model are obtained by training based on a YOLOv5 model, and the behavior recognition model is obtained by training based on a SlowFast model. According to the method, the human body detection model, the behavior recognition model and the cigarette target detection model obtained by training based on the YOLOv5 model and the SlowFast model are combined with the multi-target tracking algorithm model, so that the technical problem that the existing method cannot meet the requirements of high detection efficiency, low cost and high accuracy at the same time is solved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a smoking detection method, device and electronic equipment. Background Art

[0002] Smoking in public places, workplaces or specific areas (such as hospitals, schools, kitchens, etc.) not only threatens personal health, but also causes secondhand smoke hazards to others. Traditional smoke detectors can only detect smoke, but cannot distinguish whether it is smoke produced by cigarettes or smoke from other sources, so there are problems of false alarms and missed alarms. In addition, manual supervision is costly and inefficient. To solve these problems, an intelligent system is needed that can accurately identify cigarettes and issue an alarm in time when smoking behavior is detected. There are also some other methods that use neural network model recognition, which cannot accurately identify straws, chopsticks and cigarettes, and are prone to false detection.

[0003] In summary, existing smoking detection methods cannot simultaneously meet the requirements of high detection efficiency, low cost and high accuracy. Summary of the invention

[0004] In view of this, it is necessary to provide a smoking detection method, device and electronic equipment to solve the technical problem that the existing smoking detection method cannot simultaneously meet the requirements of high detection efficiency, low cost and high accuracy.

[0005] In order to solve the above problems, on the one hand, the present invention provides a smoking detection method, comprising: Acquire a video image to be detected, input the video image into a human body detection model, and obtain a human body region image; Input the human body area image into the multi-target tracking algorithm model to obtain continuous frames of the target person; Input the continuous frames of the target person into the behavior recognition model to obtain the image frames in which the target person has the smoking action; Input the image frame showing the target person smoking into the cigarette target detection model to obtain the detection result of whether the target person is smoking; The human body detection model and the cigarette target detection model are both obtained by training based on the YOLOv5 model, and the behavior recognition model is obtained by training based on the SlowFast model.

[0006] In a possible implementation, the human body detection model and the cigarette target detection model are both trained based on the YOLOv5 model with an SELayer attention mechanism module added.

[0007] In a possible implementation, the human body detection model is trained based on the following steps: Acquire human body images in different scenes, and create a first training data set based on the human body images; Based on the first training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and data enhancement processing is performed on the first training data set during the training process to obtain a human body detection model.

[0008] In a possible implementation, the cigarette object detection model is trained based on the following steps: Acquire cigarette images in different scenes, and create a second training data set based on the cigarette images; Based on the second training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and data enhancement processing is performed on the second training data set during the training process to obtain a cigarette target detection model.

[0009] In a possible implementation, the data enhancement processing includes: Perform multi-scale cropping on the images in the training data set to be augmented to obtain a cropped data set; Flip the images in the cropped dataset at random angles to obtain a flipped dataset; Two or more images in the flipped data set are superimposed according to random transparency to obtain a training data set after data enhancement processing.

[0010] In a possible implementation, a YOLOv5 model with an SELayer attention mechanism module added is trained based on the second training data set, and data enhancement processing is performed on the second training data set during the training process to obtain a cigarette target detection model, including: Based on the second training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and during the training process, data enhancement processing is performed on the second training data set. The learning rate of the YOLOv5 model is reduced from the initial value to the minimum value using a cosine function to obtain a cigarette target detection model.

[0011] In a possible implementation, the behavior recognition model is trained based on the following steps: Initialize the SlowFast model using random initialization or transfer learning using the weights of a pre-trained model; Obtain a third training data set, and divide the third training data set into multiple batches of data sets; each batch of data sets includes multiple video frames; Input each batch of data sets into the slow path and the fast path of the initialized SlowFast model in turn, so as to exchange information between the slow path and the fast path and obtain the output result; Determine a cross entropy loss function based on the output result and the preset actual label; The weight of the initialized SlowFast model is updated based on the gradient of the cross entropy loss function, and the parameters of the initialized SlowFast model are updated to minimize the cross entropy loss function to obtain a behavior recognition model.

[0012] In a possible implementation, the smoking detection method further includes: If it is determined based on the detection result of whether the target person is smoking that the target person is smoking, an alarm is issued.

[0013] In another aspect, the present invention further provides a smoking detection device, comprising: A human body detection module is used to obtain a video image to be detected, input the video image into a human body detection model, and obtain a human body region image; A tracking module is used to input the human body area image into the multi-target tracking algorithm model to obtain continuous frames of the target person; An action recognition module is used to input the continuous frames of the target person into the behavior recognition model to obtain image frames in which the target person has a smoking action; The cigarette recognition module is used to input the image frame of the target person smoking into the cigarette target detection model to obtain the detection result of whether the target person is smoking; The human body detection model and the cigarette target detection model are both obtained by training based on the YOLOv5 model, and the behavior recognition model is obtained by training based on the SlowFast model.

[0014] On the other hand, the present invention also provides an electronic device, including a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of any one of the above-mentioned smoking detection methods.

[0015] The beneficial effects of adopting the above implementation mode are as follows: the smoking detection method, device and electronic device provided by the present invention combine the human body detection model, behavior recognition model and cigarette target detection model obtained by training based on the YOLOv5 model and the SlowFast model, and the multi-target tracking algorithm model. For the video image to be detected, human body detection is first performed, and then the detected human body area image is continuously tracked, the continuous frames of the target person are taken, and then the smoking action recognition is performed on the continuous frames, and the image frames with smoking action are performed on cigarette target detection, and finally the detection result of whether the target person is smoking is obtained; the present invention adopts a combination of multiple different neural network models to automatically detect smokers, which can improve the efficiency of smoking detection. At the same time, the human body is first identified, then the smoking action of the human body is identified, and finally the cigarette target is identified, so as to avoid the problem of false detection that is easy to occur when cigarette target detection is performed alone, and improve the accuracy of smoking detection, so that the present invention solves the technical problem that the existing smoking detection method cannot meet the requirements of high detection efficiency, low cost and high accuracy at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A process of an embodiment of the smoking detection method provided by the present invention; Figure 2 A flow chart of another embodiment of the smoking detection method provided by the present invention; Figure 3 A principle block diagram of an embodiment of a smoking detection device provided by the present invention; Figure 4 A schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0019] In the description of the embodiments of the present application, unless otherwise specified, “plurality” means two or more than two.

[0020] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or equipment comprising a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products or equipment.

[0021] The naming or numbering of the steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0022] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] The present invention provides a smoking detection method, device and electronic equipment, which are respectively described below.

[0024] like Figure 1 As shown, the present invention provides a smoking detection method, comprising: S101, obtaining a video image to be detected, and inputting the video image into a human body detection model to obtain a human body region image; S102, inputting the human body region image into a multi-target tracking algorithm model to obtain continuous frames of the target person; S103, inputting continuous frames of the target person into a behavior recognition model to obtain image frames in which the target person has a smoking action; S104, inputting the image frame showing the target person smoking into the cigarette target detection model to obtain a detection result of whether the target person is smoking; The human body detection model and the cigarette target detection model are both obtained by training based on the YOLOv5 model, and the behavior recognition model is obtained by training based on the SlowFast model.

[0025] It is understandable that the present invention proposes an innovative smoking recognition algorithm to address the above problems, and the core innovation lies in: 1. Behavior recognition model: The continuous frame action analysis model slow-fast (SlowFast) is used to identify a series of smoking-related actions, such as holding a cigarette, lighting, inhaling, and exhaling smoke. The model can accurately identify smoking behavior by comparing it with the existing smoking behavior database.

[0026] 2. Cigarette target detection model: Based on the YOLOv5 target detection model, cigarette target detection is performed on continuous action behaviors, and secondary confirmation is performed to reduce false positives caused by misidentification of behaviors.

[0027] The technical solution provided by the present invention is as follows: 1. Network architecture: reference Figure 2 The model consists of four parts: human detection model, multi-target tracking module, behavior recognition model, and cigarette target detection model. The human detection model trained by YOLOv5 is responsible for detecting human targets from the monitoring scene, the multi-target tracking module is responsible for confirming the unique human ID and outputting continuous frames of the unique ID, the behavior recognition model is responsible for identifying actions such as holding a cigarette, lighting, inhaling, and spitting smoke, and the cigarette target detection model is responsible for detecting cigarettes from the frames where cigarettes are recognized.

[0028] 2. Attention guidance mechanism: An adaptive attention guidance mechanism module is introduced into the YOLOv5 detection model to dynamically adjust the feature extraction intensity of key areas (such as hands, mouth and cigarette contact area), reduce background noise interference, and enhance the model's sensitivity to tiny gesture changes, thereby significantly improving detection accuracy in complex scenarios.

[0029] 3. Multi-target tracking module: It integrates a multi-target tracking algorithm based on deep correlation filter, which can not only track a single object, but also handle complex scenes with multiple people and multiple mobile phones, ensuring the consistency of target identity between consecutive frames in the video stream, and improving the continuity and stability of detection.

[0030] 4. Alarm logic and decision support system: Once the system identifies smoking behavior, the alarm logic will be triggered. According to the severity and duration of the smoking behavior, the most appropriate response measures will be automatically selected, such as sending an alarm to the on-site management personnel, starting the recording and video recording function to save evidence, or even directly contacting the law enforcement department.

[0031] The present invention greatly improves the recognition accuracy and real-time performance of cigarette behavior in complex scenarios through innovative deep learning models and multi-target tracking technology. The system can be widely used in the fields of public place security monitoring, personalized recommendation systems, health habit analysis, etc., and has important application value and social benefits.

[0032] The deep learning-driven smoking behavior monitoring system proposed in the present invention effectively solves the challenges of smoking behavior detection in complex scenarios in existing technologies through unique network model innovation, multi-target tracking integration, and behavioral relationship recognition, and provides advanced technical means and solutions for research and application in related fields.

[0033] In some embodiments, the human body detection model and the cigarette target detection model are both trained based on the YOLOv5 model with an SELayer attention mechanism module added.

[0034] In some embodiments, the human body detection model is trained based on the following steps: Acquire human body images in different scenes, and create a first training data set based on the human body images; Based on the first training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and data enhancement processing is performed on the first training data set during the training process to obtain a human body detection model.

[0035] It can be understood that generating a human detection model includes: The collected data is divided into training set, validation set and test set according to a certain ratio.

[0036] The training set and validation set are used to train the YOLOv5 model with the SELayer attention mechanism module added, and the trained YOLOv5 model is tested based on the test set. If the test passes, the human body detection model is obtained.

[0037] In some embodiments, the cigarette object detection model is trained based on the following steps: Acquire cigarette images in different scenes, and create a second training data set based on the cigarette images; Based on the second training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and data enhancement processing is performed on the second training data set during the training process to obtain a cigarette target detection model.

[0038] It can be understood that generating a cigarette target detection model includes: Create training data sets: Collect data on smoking in various scenarios in public, mainly monitoring scenarios. The data that needs to be labeled includes human target frames, human IDs, action behavior labels, and cigarette target frames.

[0039] The training data set is divided into a training set, a validation set, and a test set. The specific training steps and processes of the cigarette target detection model are consistent with those of the human body detection model.

[0040] In some embodiments, data enhancement processing includes: Perform multi-scale cropping on the images in the training data set to be augmented to obtain a cropped data set; Flip the images in the cropped dataset at random angles to obtain a flipped dataset; Two or more images in the flipped data set are superimposed according to random transparency to obtain a training data set after data enhancement processing.

[0041] It is understandable that data enhancement is needed when training the model, which can effectively improve the generalization ability and robustness of the model, thereby improving the performance of the model.

[0042] Specific methods of data enhancement: 1. Multi-scale cropping: randomly select a cropping scale from multiple scales, calculate the starting position and width and height of the specific cropping, and crop a fixed area in the original image. 2. Random flipping: flip the image at a random angle. 3. Mixup: superimpose two or more images with random transparency.

[0043] In some embodiments, a YOLOv5 model with an SELayer attention mechanism module added is trained based on the second training data set, and data enhancement processing is performed on the second training data set during the training process to obtain a cigarette target detection model, including: Based on the second training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and during the training process, data enhancement processing is performed on the second training data set. The learning rate of the YOLOv5 model is reduced from the initial value to the minimum value using a cosine function to obtain a cigarette target detection model.

[0044] It is understandable that adding the SELayer attention mechanism module to the YOLOv5 model can enhance the ability to extract key features, thereby improving the detection model's ability to detect target boxes such as cigarettes.

[0045] Learning rate: Use the method of dynamically adjusting the learning rate. Specifically, use the cosine annealing strategy to adjust the learning rate, so that the learning rate can be gradually reduced from the initial value to a minimum value according to the periodicity of the cosine function. This method can help the model learn quickly in the early stage of training, and gradually slow down the learning rate in the later stage, which helps the model converge better at the end of training. After the model converges, the training is completed. The ability to detect the target box is obtained.

[0046] In some embodiments, the behavior recognition model is trained based on the following steps: Initialize the SlowFast model using random initialization or transfer learning using the weights of a pre-trained model; Obtain a third training data set, and divide the third training data set into multiple batches of data sets; each batch of data sets includes multiple video frames; Input each batch of data sets into the slow path and the fast path of the initialized SlowFast model in turn, so as to exchange information between the slow path and the fast path and obtain the output result; Determine a cross entropy loss function based on the output result and the preset actual label; The weight of the initialized SlowFast model is updated based on the gradient of the cross entropy loss function, and the parameters of the initialized SlowFast model are updated to minimize the cross entropy loss function to obtain a behavior recognition model.

[0047] It is understandable that training the SlowFast model into a behavior recognition model includes: Data preparation, including: The video clips of human ID containing smoking behaviors are cut to between 15 and 30 seconds in length, including holding a cigarette, lighting, inhaling, and exhaling smoke, and the labels are behavior categories. The data is divided into training set, validation set, and test set according to a certain ratio.

[0048] Data preprocessing, including: Cutting: Videos may need to be cut into segments of fixed length to facilitate model processing; Scaling: Video frames may need to be scaled to the same resolution; Normalization: Normalize pixel values, usually converting them to values ​​between 0 and 1.

[0049] Data augmentation, including: The data diversity can be increased by random cropping, flipping or color transformation to improve the generalization ability of the model.

[0050] Model training, including: Architecture construction: According to the SlowFast architecture, a network containing slow and fast paths is built. Usually the slow path uses a deeper network structure, while the fast path uses a shallower structure.

[0051] Initialization parameters: The weights and biases of the model need to be initialized, usually using random initialization or the weights of a pre-trained model for transfer learning.

[0052] Batch Generation: Split the dataset into batches, each batch contains a certain number of video clips.

[0053] Forward propagation: Input video clips to slow and fast paths. At each layer, the slow and fast paths may exchange information, through upsampling or downsampling operations.

[0054] Loss calculation: Calculate the gap between the model output and the actual label, usually using the cross entropy loss function.

[0055] Backpropagation: Update the model’s weights based on the gradient of the loss function.

[0056] Optimizer Update: Update the model parameters using an optimization algorithm (SGD) to minimize the loss function.

[0057] Verification and testing: Regularly evaluate model performance on the validation set, adjust model parameters or stop training (early stopping method) based on the validation results. Finally, evaluate the generalization ability of the model on the test set to ensure that the model not only performs well on the training data, but also works well on unseen data.

[0058] Model tuning: Based on the verification and testing results, it may be necessary to adjust the model structure, hyperparameters, or training strategies to further improve model performance. This completes the model training for human behavior recognition.

[0059] In some embodiments, the smoking detection method further comprises: If it is determined based on the detection result of whether the target person is smoking that the target person is smoking, an alarm is issued.

[0060] It can be understood that the human body detection model trained by YOLOv5 is first used to detect the human body area and crop the human body area.

[0061] Multi-target tracking model: A multi-target tracking model is added during the inference process to assign a unique ID to each human detection target.

[0062] Perform behavior recognition on consecutive frames for each human ID and output the behavior category.

[0063] If the category is one of holding a cigarette, lighting, inhaling, and exhaling smoke, the continuous frame images of the human body are sent to the cigarette target detection model. If a cigarette is detected, an alarm is triggered. If no cigarette is detected, the alarm is canceled.

[0064] In summary, the smoking detection method provided by the present invention includes: acquiring a video image to be detected, inputting the video image into a human body detection model to obtain a human body area image; inputting the human body area image into a multi-target tracking algorithm model to obtain continuous frames of the target person; inputting the continuous frames of the target person into a behavior recognition model to obtain image frames in which the target person has smoking actions; inputting the image frames in which the target person has smoking actions into a cigarette target detection model to obtain a detection result of whether the target person is smoking; wherein the human body detection model and the cigarette target detection model are both trained based on the YOLOv5 model, and the behavior recognition model is trained based on the SlowFast model.

[0065] The smoking detection method provided by the present invention combines a human body detection model, a behavior recognition model, a cigarette target detection model and a multi-target tracking algorithm model obtained by training based on a YOLOv5 model and a SlowFast model. For a video image to be detected, human body detection is first performed, and then the detected human body region image is continuously tracked, and continuous frames of a target person are taken, and then smoking action recognition is performed on the continuous frames, and cigarette target detection is performed on the image frames with smoking action, and finally a detection result of whether the target person is smoking is obtained; the present invention adopts a combination of multiple different neural network models to automatically detect smokers, which can improve the efficiency of smoking detection. At the same time, the human body is first identified, and then the smoking action of the human body is identified, and finally the cigarette target is identified, so as to avoid the problem of easy misdetection when performing cigarette target detection alone, and improve the accuracy of smoking detection. Therefore, the present invention solves the technical problem that the existing smoking detection method cannot simultaneously meet the requirements of high detection efficiency, low cost and high accuracy.

[0066] like Figure 3 As shown, the present invention also provides a smoking detection device 300, comprising: The human body detection module 301 is used to obtain a video image to be detected, input the video image into a human body detection model, and obtain a human body region image; Tracking module 302, used to input the human body region image into the multi-target tracking algorithm model to obtain continuous frames of the target person; The action recognition module 303 is used to input the continuous frames of the target person into the behavior recognition model to obtain the image frames in which the target person has the smoking action; The cigarette recognition module 304 is used to input the image frame showing the target person smoking into the cigarette target detection model to obtain the detection result of whether the target person is smoking; The human body detection model and the cigarette target detection model are both obtained by training based on the YOLOv5 model, and the behavior recognition model is obtained by training based on the SlowFast model.

[0067] The smoking detection device provided in the above embodiment can implement the technical solution described in the above smoking detection method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above smoking detection method embodiment, which will not be repeated here.

[0068] like Figure 4 As shown, the present invention also provides an electronic device 400. The electronic device 400 includes a processor 401, a memory 402 and a display 403. Figure 4 Only some components of the electronic device 400 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0069] In some embodiments, the memory 402 may be an internal storage unit of the electronic device 400, such as a hard disk or memory of the electronic device 400. In other embodiments, the memory 402 may also be an external storage device of the electronic device 400, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 400.

[0070] Furthermore, the memory 402 may include both an internal storage unit of the electronic device 400 and an external storage device. The memory 402 is used to store application software installed in the electronic device 400 and various data.

[0071] In some embodiments, the processor 401 may be a central processing unit (CPU), a microprocessor or other data processing chip, and is used to run program codes or process data stored in the memory 402, such as the smoking detection method of the present invention.

[0072] In some embodiments, the display 403 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 403 is used to display information on the electronic device 400 and to display a visual user interface. The components 401-403 of the electronic device 400 communicate with each other via a system bus.

[0073] In some embodiments of the present invention, when the processor 401 executes the smoking detection program in the memory 402, the following steps may be implemented: Acquire a video image to be detected, input the video image into a human body detection model, and obtain a human body region image; Input the human body area image into the multi-target tracking algorithm model to obtain continuous frames of the target person; Input the continuous frames of the target person into the behavior recognition model to obtain the image frames in which the target person has the smoking action; Input the image frame showing the target person smoking into the cigarette target detection model to obtain the detection result of whether the target person is smoking; The human body detection model and the cigarette target detection model are both obtained by training based on the YOLOv5 model, and the behavior recognition model is obtained by training based on the SlowFast model.

[0074] It should be understood that: when the processor 401 executes the smoking detection program in the memory 402, in addition to the above functions, other functions may also be implemented, and the details may refer to the description of the corresponding method embodiment above.

[0075] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 400 mentioned, and the electronic device 400 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic device may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 400 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0076] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the smoking detection method provided by the above methods is implemented, and the method comprises: Acquire a video image to be detected, input the video image into a human body detection model, and obtain a human body region image; Input the human body area image into the multi-target tracking algorithm model to obtain continuous frames of the target person; Input the continuous frames of the target person into the behavior recognition model to obtain the image frames in which the target person has the smoking action; Input the image frame showing the target person smoking into the cigarette target detection model to obtain the detection result of whether the target person is smoking; The human body detection model and the cigarette target detection model are both obtained by training based on the YOLOv5 model, and the behavior recognition model is obtained by training based on the SlowFast model.

[0077] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0078] The smoking detection method, device and electronic device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A smoking detection method, characterized in that: include: Acquire a video image to be detected, input the video image into a human body detection model, and obtain a human body region image; Input the human body area image into the multi-target tracking algorithm model to obtain continuous frames of the target person; Input the continuous frames of the target person into the behavior recognition model to obtain the image frames in which the target person has the smoking action; Input the image frame showing the target person smoking into the cigarette target detection model to obtain the detection result of whether the target person is smoking; The human body detection model and the cigarette target detection model are both obtained by training based on the YOLOv5 model, and the behavior recognition model is obtained by training based on the SlowFast model.

2. The smoking detection method according to claim 1, characterized in that: The human body detection model and the cigarette target detection model are both trained based on the YOLOv5 model with the SELayer attention mechanism module added.

3. The smoking detection method according to claim 2, characterized in that: The human body detection model is trained based on the following steps: Acquire human body images in different scenes, and create a first training data set based on the human body images; Based on the first training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and data enhancement processing is performed on the first training data set during the training process to obtain a human body detection model.

4. The smoking detection method according to claim 3, characterized in that: The cigarette target detection model is trained based on the following steps: Acquire cigarette images in different scenes, and create a second training data set based on the cigarette images; Based on the second training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and data enhancement processing is performed on the second training data set during the training process to obtain a cigarette target detection model.

5. The smoking detection method according to claim 4, characterized in that: Data enhancement processing, including: Perform multi-scale cropping on the images in the training data set to be augmented to obtain a cropped data set; Flip the images in the cropped dataset at random angles to obtain a flipped dataset; Two or more images in the flipped data set are superimposed according to random transparency to obtain a training data set after data enhancement processing.

6. The smoking detection method according to claim 4, characterized in that: Based on the second training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and during the training process, data enhancement processing is performed on the second training data set to obtain a cigarette target detection model, including: Based on the second training data set, a YOLOv5 model with an SELayer attention mechanism module added is trained, and during the training process, data enhancement processing is performed on the second training data set. The learning rate of the YOLOv5 model is reduced from the initial value to the minimum value using a cosine function to obtain a cigarette target detection model.

7. The smoking detection method according to claim 1, characterized in that: The behavior recognition model is trained based on the following steps: Initialize the SlowFast model using random initialization or transfer learning using the weights of a pre-trained model; Obtain a third training data set, and divide the third training data set into multiple batches of data sets; each batch of data sets includes multiple video frames; Input each batch of data sets into the slow path and the fast path of the initialized SlowFast model in turn, so as to exchange information between the slow path and the fast path and obtain the output result; Determine a cross entropy loss function based on the output result and the preset actual label; The weight of the initialized SlowFast model is updated based on the gradient of the cross entropy loss function, and the parameters of the initialized SlowFast model are updated to minimize the cross entropy loss function to obtain a behavior recognition model.

8. The smoking detection method according to any one of claims 1 to 7, characterized in that: Also includes: If it is determined based on the detection result of whether the target person is smoking that the target person is smoking, an alarm is issued.

9. A smoking detection device, characterized in that: include: A human body detection module is used to obtain a video image to be detected, input the video image into a human body detection model, and obtain a human body region image; A tracking module is used to input the human body area image into the multi-target tracking algorithm model to obtain continuous frames of the target person; An action recognition module is used to input the continuous frames of the target person into the behavior recognition model to obtain image frames in which the target person has a smoking action; The cigarette recognition module is used to input the image frame of the target person smoking into the cigarette target detection model to obtain the detection result of whether the target person is smoking; The human body detection model and the cigarette target detection model are both obtained by training based on the YOLOv5 model, and the behavior recognition model is obtained by training based on the SlowFast model.

10. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory, and is used to execute the program stored in the memory to implement the steps of the smoking detection method according to any one of claims 1 to 8.

Images