Public place indoor smoking identification and solution system
By using a system combining data acquisition and mobile smart devices in public places, using the YOLOv5 model for smoking detection, the problems of low detection efficiency and accuracy in the existing technology are solved, and accurate identification and timely warning of smokers are achieved, and the improvement of smoking personnel is improved. Detection accuracy and efficiency.
Patent Information
- Application Number
- CN202410409156.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-07
- Publication Date
- 2025-01-10
AI Technical Summary
The prior art has low efficiency and low accuracy for indoor smoking detection in public places, and mostly relies on conscious or tobacco control volunteers, and lacks an effective identification and solution system.
The data acquisition module is used to obtain indoor smoking image data, mark and divide the training set and verification set through the labelilimg tool, build a target detection model, combine the camera and smoke detector of mobile smart devices, use the YOLOv5 model to perform smoking detection, and issue it based on the detection results. warn.
It improves the rate and accuracy of smoking identification, reduces costs, realizes timely warnings to smokers, and effectively solves the problem of indoor smoking.
Smart Images
Figure CN119296020A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of smart city technology, and in particular to a system for identifying and resolving indoor smoking in public places. Background Art
[0002] To protect public health, many cities have issued corresponding laws and regulations, stipulating that smoking is completely banned in indoor public places, indoor workplaces, and public transportation. Although there are many object detection algorithms that can detect smoking behavior at this stage, there are few smoking identification devices in public places on the market. Smoking bans mostly rely on self-consciousness or smoking control volunteers, and existing object detection algorithms have problems with low detection efficiency and low detection accuracy. Summary of the invention
[0003] The present invention provides a system for identifying and solving indoor smoking in public places to overcome the above technical problems.
[0004] A system for identifying and solving indoor smoking in public places includes a data collection module, a data import module, a server module and a data export module.
[0005] The data acquisition module is used to obtain an indoor smoking image dataset, and to label the images in the indoor smoking image dataset based on the labelimg tool. The labeling of the images in the indoor smoking image dataset based on the labelimg tool is to label the smoking behavior in the image. The smoking behavior includes taking a cigarette and not taking a cigarette. The taking a cigarette includes taking a cigarette but not smoking and taking a cigarette and smoking. The labeled indoor smoking image dataset is divided into a training set and a verification set, a target detection model is constructed, the target detection model is trained based on the training set, and the trained target detection model is verified based on the verification set.
[0006] The data import module is used to install a camera, an ultrasonic rangefinder and a smoke detector on the mobile smart device, obtain the indoor video data stream through the camera on the mobile smart device, obtain the indoor smoke environment information through the smoke detector on the mobile smart device, and transmit the video data stream and the smoke environment information to the server module respectively.
[0007] The server module is used to store video data streams and smoke environment information, and judge whether someone is smoking indoors based on the video data streams and smoke environment information at the current moment. If someone is smoking indoors, the number and position of smokers are obtained. If not, it is re-judged whether someone is smoking indoors at the next moment. The server module includes a big data storage submodule and an algorithm recognition submodule. The big data storage submodule is used to store video data streams and smoke environment information, and regularly perform manual calibration on the stored video data streams, and perform correction training on the verified target detection model based on the manually calibrated video data set. The algorithm recognition submodule is used to perform smoking detection on the video data stream at the current moment based on the corrected trained target detection model and obtain the detection result. The detection result is an image marked with a cigarette holder and the number and bounding box coordinates of the cigarette holder. The bounding box coordinates are used to represent the coordinates of the cigarette holding behavior in the marked image. The number and position of smokers are determined based on the detection result and the smoke environment information at the current moment, and the number and position of smokers are transmitted to the data export module.
[0008] The data export module is used to control the mobile smart device to track and warn smokers based on their locations, and to display the data on the front-end interface based on the number and location of smokers, while performing statistical analysis based on the number and location of historical smokers.
[0009] Preferably, the method of determining the number and positions of smokers based on the detection results and the smoke environment information at the current moment includes setting a weight corresponding to the smoke environment information at the current moment, multiplying the smoke environment information at the current moment by the weight as the smoking probability, and judging whether the smoking probability exceeds a threshold value. If it exceeds the threshold value, the images marked with cigarettes in the detection results are classified again according to the corrected trained target detection model. The classification of the images marked with cigarettes in the detection results includes dividing the images marked with cigarettes into two categories: holding a cigarette and smoking, and holding a cigarette but not smoking. The number of images classified as holding a cigarette and smoking is used as the number of smokers, and the position of the smoker is calculated according to the bounding box coordinates corresponding to the images holding a cigarette and smoking. Otherwise, it indicates that no one is smoking.
[0010] Preferably, controlling the mobile smart device to track and warn the smoker according to his / her location includes:
[0011] Step 1: Randomly select a smoker as the target point.
[0012] Step 2: Track the target point according to the target tracking strategy. Tracking the target point according to the target tracking strategy includes converting the position of the target point into coordinates relative to the center of the camera field of view according to formulas (1) and (2).
[0013] target_relative_x=x1-x / 2 (1)
[0014] target_relative_y=y1-y / 2 (2)
[0015] Among them, (x1, y1) is the position of the target point, (x, y) represents the resolution width and height of the camera, (target_relative_x, target_relative_y) represents the coordinates of the target point relative to the center of the camera's field of view,
[0016] According to formulas (3) and (4), the horizontal deflection angle a and vertical deflection angle b of the mobile smart device are calculated.
[0017] a=(target_relative_x / (x / 2))*(HFOV / 2) (3)
[0018] b=(target_relative_y / (y / 2))*(VFOV / 2) (4)
[0019] Among them, HFOV represents the horizontal field of view of the camera, and VFOV represents the vertical field of view of the camera.
[0020] The mobile smart device deflects according to the horizontal and vertical deflection angles, moves toward the smoker to track him and issues a sound warning.
[0021] Step 3: Determine whether all smokers have been traversed. If so, end the tracking and warning. If not, select a smoker who is closest to the mobile smart device from the remaining smokers who have not been traversed as the target point, and return to step 2.
[0022] Preferably, the target detection model is a YOLOv5 model, the YOLOv5 model is trained based on a training set, and the trained YOLOv5 model is verified based on a verification set.
[0023] Preferably, the setting of the weight corresponding to the smoke environment information at the current moment includes setting different weights according to different application scenarios.
[0024] The present invention provides a system for identifying and solving indoor smoking in public places. It adopts the YOLO algorithm to replace manual identification of smoking behavior and other target detection algorithms, reduces human misjudgment, and improves the speed and accuracy of identifying smokers. It also combines image acquisition equipment with smoke detection equipment, greatly reduces costs, improves detection accuracy, and can issue sound warnings to smokers in a timely manner, thereby effectively solving the problem of indoor smoking. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0026] Figure 1 It is a system structure diagram of the present invention;
[0027] Figure 2 This is a functional diagram of the indoor smoking identification and solution in public places of the present invention. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0029] Figure 1 It is a structural diagram of the system of the present invention, such as Figure 1 As shown, the method of this embodiment may include:
[0030] A system for identifying and solving indoor smoking in public places includes a data collection module, a data import module, a server module and a data export module.
[0031] The data acquisition module is used to obtain an indoor smoking image dataset, and to label the images in the indoor smoking image dataset based on the labelimg tool. The labeling of the images in the indoor smoking image dataset based on the labelimg tool is to label the smoking behavior in the image. The smoking behavior includes taking a cigarette and not taking a cigarette. The taking a cigarette includes taking a cigarette but not smoking and taking a cigarette and smoking. The labeled indoor smoking image dataset is divided into a training set and a verification set, a target detection model is constructed, the target detection model is trained based on the training set, and the trained target detection model is verified based on the verification set.
[0032] The data import module is used to install a camera, an ultrasonic rangefinder and a smoke detector on the mobile smart device, obtain the indoor video data stream through the camera on the mobile smart device, obtain the indoor smoke environment information through the smoke detector on the mobile smart device, and transmit the video data stream and the smoke environment information to the server module respectively.
[0033] The server module is used to store video data streams and smoke environment information, and judge whether someone is smoking indoors based on the video data streams and smoke environment information at the current moment. If someone is smoking indoors, the number and position of smokers are obtained. If not, it is re-judged whether someone is smoking indoors at the next moment. The server module includes a big data storage submodule and an algorithm recognition submodule. The big data storage submodule is used to store video data streams and smoke environment information, and regularly perform manual calibration on the stored video data streams, and perform correction training on the verified target detection model based on the manually calibrated video data set. The algorithm recognition submodule is used to perform smoking detection on the video data stream at the current moment based on the corrected trained target detection model and obtain the detection result. The detection result is an image marked with a cigarette holder and the number and bounding box coordinates of the cigarette holder. The bounding box coordinates are used to represent the coordinates of the cigarette holding behavior in the marked image. The number and position of smokers are determined based on the detection result and the smoke environment information at the current moment, and the number and position of smokers are transmitted to the data export module.
[0034] The data export module is used to control the mobile smart device to track and warn smokers according to their locations, and to display the numbers and locations of smokers on the front-end interface, while performing statistical analysis based on the numbers and locations of smokers in history.
[0035] Specifically, the server module can be deployed locally or in the cloud according to user needs, and can provide target detection models of different sizes according to user levels; provide targeted data analysis interfaces according to user needs, such as: the number of illegal smoking in key areas during key time periods, etc.; customize the front-end display interface according to user needs. By combining target detection with traditional sensor detection, the possibility of misjudgment can be reduced as much as possible. A simple target detection algorithm may have false detection (just taking a cigarette but not smoking), and traditional indoor smoke sensors in cafeterias and other places will also falsely alarm when no one is smoking. Therefore, the two are combined, and users can reasonably set weights according to their needs.
[0036] Based on the above scheme, this embodiment uses the YOLO algorithm to replace manual identification of smoking behavior and other target detection algorithms, reducing human misjudgment and improving the speed and accuracy of identifying smokers. It also combines image acquisition equipment with smoke detection equipment, greatly reducing costs and improving detection accuracy. At the same time, it can issue sound warnings to smokers in a timely manner, thereby effectively solving the problem of indoor smoking.
[0037] Specifically, a system for identifying and solving indoor smoking in public places includes a data collection module, a data import module, a server module and a data export module.
[0038] The data acquisition module is used to obtain an indoor smoking image dataset, and to label the images in the indoor smoking image dataset based on the labelimg tool. The labeling of the images in the indoor smoking image dataset based on the labelimg tool is to label the smoking behavior in the image. The smoking behavior includes taking a cigarette and not taking a cigarette. The taking a cigarette includes taking a cigarette but not smoking and taking a cigarette and smoking. The labeled indoor smoking image dataset is divided into a training set and a validation set, and a target detection model is constructed. The target detection model is trained based on the training set, and the trained target detection model is validated according to the validation set. Target detection is to locate and identify the target of interest in a picture. After 2014, in particular, the target detection algorithm based on deep learning has developed rapidly. The YOLO series algorithm is a typical one-stage target detection algorithm, which combines the classification with the regression problem of target positioning by using the anchor box, thereby achieving high efficiency, flexibility and good generalization performance. YOLOv5 is the fifth generation of the single-stage target detection algorithm YOLO. According to the experimental conclusion, it has been significantly improved in terms of speed and accuracy.
[0039] The data import module is used to install a camera, an ultrasonic rangefinder and a smoke detector on the mobile smart device, obtain the indoor video data stream through the camera on the mobile smart device, obtain the indoor smoke environment information through the smoke detector on the mobile smart device, and transmit the video data stream and the smoke environment information to the server module respectively.
[0040] The server module is used to store video data streams and smoke environment information, and judge whether someone is smoking indoors based on the video data streams and smoke environment information at the current moment. If someone is smoking indoors, the number and position of smokers are obtained. If not, it is re-judged whether someone is smoking indoors at the next moment. The server module includes a big data storage submodule and an algorithm recognition submodule. The big data storage submodule is used to store video data streams and smoke environment information, and regularly perform manual calibration on the stored video data streams, and perform correction training on the verified target detection model based on the manually calibrated video data set. The algorithm recognition submodule is used to perform smoking detection on the video data stream at the current moment based on the corrected trained target detection model and obtain the detection result. The detection result is an image marked with a cigarette holder and the number and bounding box coordinates of the cigarette holder. The bounding box coordinates are used to represent the coordinates of the cigarette holding behavior in the marked image. The number and position of smokers are determined based on the detection result and the smoke environment information at the current moment, and the number and position of smokers are transmitted to the data export module.
[0041] The method of determining the number and position of smokers based on the detection results and the smoke environment information at the current moment includes setting a weight corresponding to the smoke environment information at the current moment, multiplying the smoke environment information at the current moment by the weight as the smoking probability, and judging whether the smoking probability exceeds a threshold value. If it exceeds the threshold value, the images marked with cigarettes in the detection results are classified again according to the corrected trained target detection model. The classification of the images marked with cigarettes in the detection results is to divide the images marked with cigarettes into two categories: holding cigarettes and smoking, and holding cigarettes but not smoking. The number of images classified as holding cigarettes and smoking is used as the number of smokers, and the position of the smoker is calculated according to the bounding box coordinates corresponding to the images of holding cigarettes and smoking. Otherwise, it means that no one is smoking.
[0042] The weight corresponding to the smoke environment information at the current moment is set to include setting different weights according to different application scenarios. For example, when the application scenario is an indoor canteen, the weight value is adjusted lower, and when the application scenario is an indoor corridor or a living room, the weight value is adjusted higher.
[0043] The number and location of smokers are then transferred to the data export module.
[0044] The data export module is used to control the mobile smart device to track and warn the smoker according to the location of the smoker, and the control of the mobile smart device to track and warn the smoker according to the location of the smoker includes:
[0045] Step 1: Randomly select a smoker as the target point.
[0046] Step 2: Track the target point according to the target tracking strategy. Tracking the target point according to the target tracking strategy includes converting the position of the target point into coordinates relative to the center of the camera field of view according to formulas (1) and (2).
[0047] target_relative_x=x1-x / 2 (1)
[0048] target_relative_y=y1-y / 2 (2)
[0049] Among them, (x1, y1) is the position of the target point, (x, y) represents the resolution width and height of the camera, (target_relative_x, target_relative_y) represents the coordinates of the target point relative to the center of the camera's field of view,
[0050] According to formulas (3) and (4), the horizontal deflection angle a and vertical deflection angle b of the mobile smart device are calculated.
[0051] a=(target_relative_x / (x / 2))*(HFOV / 2) (3)
[0052] b=(target_relative_y / (y / 2))*(VFOV / 2) (4)
[0053] Among them, HFOV represents the horizontal field of view of the camera, and VFOV represents the vertical field of view of the camera.
[0054] Specifically, assume that the camera is on a two-dimensional plane, with the center of its field of view as the origin, and width and height as x, y. The target point is on the same plane, with a center point (x1, y1). The deflection angles we need to calculate are the horizontal angle a and the vertical angle b. We assume that the field of view (FOV) of the camera is known, which is usually determined by the specifications of the camera.
[0055] The mobile smart device deflects according to the horizontal and vertical deflection angles, moves toward the smoker to track him and issues a sound warning.
[0056] Step 3: Determine whether all smokers have been traversed. If so, end the tracking and warning. If not, select a smoker who is closest to the mobile smart device from the remaining smokers who have not been traversed as the target point, and return to step 2.
[0057] The number and location of smokers are displayed on the front-end interface, and statistical analysis is performed based on the number and location of historical smokers. Specifically, this embodiment uses technologies such as Vue to build a page display module. Vue is a progressive framework for building user interfaces, which greatly simplifies Web development. The project management and user ends use the Vue framework, and mainly use the Element UI framework for page beautification.
[0058] Specifically, this embodiment also includes indoor smoking identification and solutions in public places, which can be divided into a user end and a management end, such as Figure 2 As shown, the user-side functions include user login and registration, monitoring and viewing, jurisdiction change application, push image management, personal information maintenance, and viewing statistical information. The management-side functions include user login, monitoring and viewing, jurisdiction management, hardware management, early warning behavior management, marking key screenshots, user management, and statistical information management.
[0059] Overall beneficial effects:
[0060] The present invention provides a system for identifying and solving indoor smoking in public places. It adopts the YOLO algorithm to replace manual identification of smoking behavior and other target detection algorithms, reduces human misjudgment, and improves the speed and accuracy of identifying smokers. It also combines image acquisition equipment with smoke detection equipment, greatly reduces costs, improves detection accuracy, and can issue sound warnings to smokers in a timely manner, thereby effectively solving the problem of indoor smoking.
[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A system for identifying and solving indoor smoking in public places, characterized in that: Including data acquisition module, data import module, server module and data export module, The data acquisition module is used to obtain an indoor smoking image dataset, and to label the images in the indoor smoking image dataset based on the labelimg tool. The labeling of the images in the indoor smoking image dataset based on the labelimg tool is to label the smoking behavior in the image. The smoking behavior includes taking a cigarette and not taking a cigarette. The taking a cigarette includes taking a cigarette but not smoking and taking a cigarette and smoking. The labeled indoor smoking image dataset is divided into a training set and a verification set, a target detection model is constructed, the target detection model is trained based on the training set, and the trained target detection model is verified based on the verification set. The data import module is used to obtain detection data of the mobile smart device, and the detection data includes video data stream and smoke environment information. The mobile smart device is equipped with a camera, an ultrasonic rangefinder and a smoke detector. The indoor video data stream is obtained through the camera on the mobile smart device, and the indoor smoke environment information is obtained through the smoke detector on the mobile smart device. The video data stream and the smoke environment information are transmitted to the server module respectively. The server module is used to store video data streams and smoke environment information, and judge whether someone is smoking indoors based on the video data streams and smoke environment information at the current moment. If someone is smoking indoors, the number and position of smokers are obtained. If not, it is re-judged whether someone is smoking indoors at the next moment. The server module includes a big data storage submodule and an algorithm recognition submodule. The big data storage submodule is used to store video data streams and smoke environment information, and regularly perform manual calibration on the stored video data streams, and perform correction training on the verified target detection model based on the manually calibrated video data set. The algorithm recognition submodule is used to perform smoking detection on the video data stream at the current moment based on the corrected trained target detection model and obtain the detection result. The detection result is an image marked with a cigarette holder and the number and bounding box coordinates of the cigarette holder. The bounding box coordinates are used to represent the coordinates of the cigarette holding behavior in the marked image. The number and position of smokers are determined based on the detection result and the smoke environment information at the current moment, and the number and position of smokers are transmitted to the data export module. The data export module is used to control the mobile smart device to track and warn smokers based on their locations, and to display the data on the front-end interface based on the number and location of smokers, while performing statistical analysis based on the number and location of historical smokers.
2. A system for identifying and solving indoor smoking in public places according to claim 1, characterized in that: The method of determining the number and position of smokers based on the detection results and the smoke environment information at the current moment includes setting a weight corresponding to the smoke environment information at the current moment, multiplying the smoke environment information at the current moment by the weight as the smoking probability, and judging whether the smoking probability exceeds a threshold value. If it exceeds the threshold value, the images marked with cigarettes in the detection results are classified again according to the corrected trained target detection model. The classification of the images marked with cigarettes in the detection results is to divide the images marked with cigarettes into two categories: holding cigarettes and smoking, and holding cigarettes but not smoking. The number of images classified as holding cigarettes and smoking is used as the number of smokers, and the position of the smoker is calculated according to the bounding box coordinates corresponding to the images of holding cigarettes and smoking. Otherwise, it means that no one is smoking.
3. A system for identifying and solving indoor smoking in public places according to claim 1, characterized in that: The controlling of the mobile smart device to track and warn the smoker according to his / her location includes: Step 1: Randomly select a smoker as the target point. Step 2: Track the target point according to the target tracking strategy. Tracking the target point according to the target tracking strategy includes converting the position of the target point into coordinates relative to the center of the camera field of view according to formulas (1) and (2). target_relative_x=x1-x / 2(1) target_relative_y = y1 - y / 2 (2) Among them, (x1, y1) is the position of the target point, (x, y) represents the resolution width and height of the camera, (target_relative_x, target_relative_y) represents the coordinates of the target point relative to the center of the camera's field of view, According to formulas (3) and (4), the horizontal deflection angle a and vertical deflection angle b of the mobile smart device are calculated. a=(target_relative_x / (x / 2))*(HFOV / 2)(3) b=(target_relative_y / (y / 2))*(VFOV / 2)(4) Among them, HFOV represents the horizontal field of view of the camera, and VFOV represents the vertical field of view of the camera. The mobile smart device deflects according to the horizontal and vertical deflection angles, moves toward the smoker to track him and issues a sound warning. Step 3: Determine whether all smokers have been traversed. If so, end the tracking and warning. If not, select a smoker who is closest to the mobile smart device from the remaining smokers who have not been traversed as the target point, and return to step 2.
4. A system for identifying and solving indoor smoking in public places according to claim 1, characterized in that: The target detection model is a YOLOv5 model, which is trained based on a training set and verified based on a verification set.
5. The system for identifying and solving indoor smoking in public places according to claim 2 is characterized in that: The setting of the weight corresponding to the smoke environment information at the current moment includes setting different weights according to different application scenarios.
Citation Information
Cited By
Civil engineering field fire risk intelligent assessment system and method
CN121640396A