Smoking behavior detection method based on YOLOv5s and robot

Abstract

The invention discloses a smoking behavior detection method based on YOLOv5s and a robot, and belongs to the field of target detection.The smoking behavior detection method comprises the steps that a smoking behavior data set is collected, and the collected data set is labeled through labmeli; dividing the labeled data set to generate a training set, a verification set and a test set; a data set optimization strategy is adopted, and a Yolov5s recognition model is trained according to the generated training set, verification set and test set; the method comprises the following steps of: deploying a trained Yov5s identification model on robot equipment, deploying TensorRT in the robot equipment, acquiring field image data through a camera on a robot, performing real-time reasoning detection by using the deployed YOLOv5s-TensorRT, and transmitting a processing signal to an MCU (Microprogrammed Control Unit) through a UART (Universal Asynchronous Receiver / Transmitter) when a smoking behavior is detected so as to trigger a buzzer to give an alarm. The problems that existing smoking monitoring is poor in flexibility, cloud computing performance is bottleneck, and a complex dynamic environment cannot be automatically adapted are solved.

Description

Technical Field

[0001] This invention belongs to the field of target detection, specifically relating to a smoking behavior detection method and robot based on YOLOv5s. Background Technology

[0002] With the increasing number of smokers, health hazards and safety accidents caused by smoking are occurring frequently, making effective supervision of smoking in key areas such as public places and industrial sites an urgent need. Currently, the mainstream existing technologies for monitoring smoking mainly fall into the following two categories:

[0003] (1) Manual monitoring technology based on fixed-point cameras

[0004] This is currently the most common monitoring method in public places and work environments: surveillance cameras are deployed at fixed locations, and the images are transmitted to a monitoring center via a video surveillance system, where human staff observe in real time to identify whether anyone is smoking. While this method is technically mature and widely applicable, its effectiveness depends entirely on the focus and responsibility of the monitoring personnel, leading to problems such as high workload, fatigue leading to oversights, low detection efficiency, and the potential for blind spots. Furthermore, the installation locations of fixed cameras are limited and difficult to adjust flexibly, failing to meet the dynamic monitoring needs of large-scale, ever-changing scenarios.

[0005] (2) Visual inspection system based on fixed monitoring and artificial intelligence

[0006] With the development of deep learning-based visual recognition technology, some existing technologies integrate AI visual algorithms (such as YOLO and SSD) into fixed monitoring systems to automatically identify smoking behavior in camera footage. Compared to purely manual observation, these technologies offer significant improvements in detection accuracy and efficiency. However, current applications are mostly concentrated indoors or in single scenarios, and the systems still rely on fixed cameras, limiting the field of view and failing to cover complex and ever-changing monitoring environments. Furthermore, the increased number of cameras significantly raises deployment and maintenance costs and leads to a surge in the number of images, increasing the burden on backend processing.

[0007] In summary, existing smoking behavior monitoring technologies mainly rely on fixed-point cameras combined with human monitoring. Although some solutions have introduced artificial intelligence visual detection algorithms, the following technical problems still exist:

[0008] (1) Fixed monitoring points are rigid and prone to creating blind spots.

[0009] In existing technologies, cameras are installed in fixed locations, which makes it difficult to adapt to dynamic, complex, or large-area scenes, resulting in limited monitoring range. A large number of cameras need to be installed to cover the entire location, significantly increasing hardware and wiring maintenance costs, while blind spots may still exist.

[0010] (2) Cloud AI computing is highly dependent, bandwidth pressure is high, and inference latency is relatively high.

[0011] Existing AI-based smoking behavior detection systems typically deploy model inference and computation in the cloud, resulting in the need to upload large amounts of on-site data. This leads to issues such as high bandwidth requirements, significant network fluctuations, and high data transmission latency, making it unsuitable for security monitoring scenarios that require rapid response.

[0012] (3) Traditional regulatory systems have limited intelligence and lack dynamic obstacle avoidance and autonomous navigation capabilities.

[0013] Most existing regulatory systems are fixed or rely on manual inspections, which cannot adapt to the regulatory needs in complex and dynamic environments. They are affected by changes in the scene (such as personnel movement and obstacle movement), resulting in poor regulatory continuity. Summary of the Invention

[0014] To overcome the shortcomings of the existing technology, the present invention aims to provide a smoking behavior detection method and robot based on YOLOv5s, which solves the problems of poor flexibility, cloud computing performance bottleneck, and inability to autonomously adapt to complex dynamic environments in existing smoking monitoring.

[0015] This invention is achieved through the following technical solution:

[0016] A method for detecting smoking behavior based on YOLOv5s includes the following steps:

[0017] S1: Collect a dataset of smoking behavior and label the collected dataset using labelimg;

[0018] S2: Divide the dataset labeled in step S1 to generate training set, validation set and test set;

[0019] S3: Using a dataset optimization strategy, train the Yolov5s recognition model based on the training set, validation set, and test set generated in step S2;

[0020] S4: Deploy the trained YOLOv5s recognition model on the robot device, and also deploy TensorRT on the robot device. Collect on-site image data through the camera on the robot, and use the deployed YOLOv5s-TensorRT for real-time inference detection. When smoking behavior is detected, transmit the processing signal to the MCU through UART, thereby triggering the buzzer alarm.

[0021] Furthermore, in step S1, the dataset includes images and videos from various environments where smoking behavior needs to be detected.

[0022] Furthermore, the elements labeled by labelimg in step S1 include the smoker's posture, the glow and smoke of the cigarette butt, the color and shape of the cigarette butt, and the smoking gesture.

[0023] Furthermore, in step S2, the training set accounts for 70%, the validation set accounts for 20%, and the test set accounts for 10%. The training set is stored in the train file of the detection directory, the validation set is stored in the val file of the detection directory, and the test set is stored in the test file of the detection directory.

[0024] Further, step S3 specifically includes:

[0025] S3.1: Configure a Python virtual environment using Anaconda. Open the weight file in VS Code, configure the Python virtual environment configured by Anaconda, execute the command to activate the virtual environment in the VS Code command window, and install all necessary virtual environment libraries.

[0026] S3.2: Employing a dataset optimization strategy, data from the training, validation, and test sets are read. Mosaic data augmentation and HSV color perturbation are applied to enhance the images. The Mosaic data augmentation strategy includes brightness adjustment, rotation and translation, scaling, noise addition, random cropping, and color transformation operations on the training data. Brightness adjustment simulates smoking behavior images under different lighting conditions; rotation and translation simulate the posture changes of smokers at different angles and positions; scaling simulates smoking targets captured by surveillance cameras at different distances; noise addition simulates image interference that may occur in actual surveillance scenarios; random cropping simulates partial occlusion or limited field of view; and HSV color perturbation simulates visual changes under different weather or lighting conditions.

[0027] S3.3: Train the yolov5s visual recognition model using the enhanced training set, validation set, and test set. The training results are stored in the runs file.

[0028] Furthermore, during the training process in step S3, the training epochs are set to 500, the batch size to 32, the initial learning rate to 0.01, the momentum coefficient to 0.937, the weight decay coefficient to 0.0005, the initial training epoch warmup_epochs to 4, the hue, saturation, and brightness to 0.015, 0.07, and 0.4 respectively, the random angle enhancement to 5, the translation enhancement ratio to 0.15, the random scaling range to 1, the slight clipping angle to 3, the horizontal flip probability to 0.5, and the automatic termination epoch early_stopping to 20.

[0029] A robot comprising the aforementioned YOLOv5s-based smoking behavior detection method.

[0030] Furthermore, the robot incorporates the Cartographer mapping algorithm and the ROS navigation system.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] This invention trains the YOLOv5s deep learning smoking behavior detection model and accelerates it using the TensorRT inference engine and CUDA / cuDNN GPU, enabling efficient local inference directly on-site. This reduces network bandwidth requirements, significantly shortens response time, and achieves real-time inference, overcoming the high dependence on cloud computing and bandwidth and latency issues of existing technologies. Furthermore, it innovatively introduces robots to replace the traditional fixed monitoring mode, solving the problems of poor regulatory flexibility, numerous blind spots, and weak scene adaptability in existing technologies. Combined with the Cartographer mapping algorithm and ROS Navigation autonomous navigation and obstacle avoidance function, it comprehensively improves the intelligence, flexibility, and efficiency of smoking behavior monitoring.

[0033] This invention introduces multiple data augmentation techniques into the dataset optimization strategy, effectively improving the model's robustness and generalization ability in complex real-world scenarios, and enhancing its recognition of smoking behavior under different environmental, angle, occlusion, and interference conditions. Simultaneously, during model training, the training epochs, batch size, initial learning rate, momentum coefficient, and weight decay parameters are appropriately set, and a learning rate warmup mechanism and early stopping strategy are introduced, effectively improving the model's convergence speed and stability, and preventing overfitting.

[0034] The robot provided by this invention can replace traditional fixed-point cameras, enabling omnidirectional movement and flexible deployment, effectively eliminating blind spots, improving the flexibility of site supervision, and by introducing the Cartographer mapping algorithm and ROSNAggation function package into the robot, the robot has the ability to navigate autonomously and avoid obstacles dynamically, realizing intelligent and continuous site supervision. Attached Figure Description

[0035] Figure 1 This is a flowchart of the method of the present invention;

[0036] Figure 2 This is a visualization of the target bounding box distribution for smoking behavior detection according to the present invention.

[0037] Figure 3 This is a graph showing the metrics of the training process for the smoking behavior detection model of the present invention.

[0038] Figure 4 This is a comparison chart of frame rates before and after optimization in this invention;

[0039] Figure 5 This is a comparison chart of CPU utilization before and after optimization in this invention;

[0040] Figure 6 The diagram shows the navigation effects before and after optimization according to the present invention. Detailed Implementation

[0041] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0042] This invention provides a smoking behavior detection method and robot based on YOLOv5s. On one hand, it employs the YOLOv5s deep learning object detection algorithm. By collecting a smoking behavior dataset and training a model, the system can automatically and accurately identify smoking behavior in images. The trained model is deployed on an NVIDIA Jetson Nano edge computing device, combined with the TensorRT inference engine, CUDA, and cuDNN GPU acceleration technologies to achieve rapid inference from real-time images, improving detection efficiency. On the other hand, it innovatively introduces a mobile patrol sentry robot to replace traditional fixed cameras for flexible and mobile surveillance. The robot uses an STM32 microcontroller as its main control board, driving four Mecanum wheel motors for omnidirectional movement. The patrol robot can perform efficient multi-sensor fusion mapping using the Cartographer algorithm and achieve autonomous navigation and obstacle avoidance patrolling using the ROSNavigation function package. It also supports remote-controlled patrolling, adapting to the surveillance needs of different application scenarios and effectively solving the problem of real-time and accurate detection of smoking behavior in various environments requiring strict control of open flames.

[0043] First, the environment is deployed. The lightweight YOLOv5S is deployed on a PC via GitHub, and also on a mobile edge device, namely a mobile robot equipped with Jetson, as detailed below:

[0044] Obtain the relevant YOLOv5s files (mainly the weight files) from GitHub, install Anaconda and configure the Python environment. Similarly, deploy YOLOv5s and the Python environment (the versions must be the same) on the Jetson nano device, and deploy TensorRT on the Jetson nano to prepare for subsequent optimizations.

[0045] Place the labeled dataset into the yolov5s detection directory, and perform a brief test on the PC. By running detect.py, check whether the camera is on and what the frame rate is. The goal is to achieve data stream acquisition from the camera.

[0046] like Figure 1 As shown, a smoking behavior detection method based on YOLOv5s includes the following steps:

[0047] S1: Collect a smoking behavior dataset, label the collected dataset, and accurately label the identified objects. This process is performed in the object detection and labeling tool labelimg.

[0048] S2: Further screening and partitioning of the labeled dataset, and generating training, validation and test sets;

[0049] S3: Employ a dataset optimization strategy to train a lightweight Yolov5s recognition model using the divided training, validation, and test sets;

[0050] S4: Deploy the trained YOLOv5s recognition model on the robot device, and also deploy TensorRT on the robot device. Collect on-site image data through the camera on the robot, and use the deployed YOLOv5s-TensorRT for real-time inference detection. When smoking behavior is detected, transmit the processing signal to the MCU through UART, thereby triggering the buzzer alarm.

[0051] Specifically, S1 consists of the following steps:

[0052] S1.1: Collect datasets in various environments that require the detection of smoking behavior, including factories, campuses, indoor and outdoor environments, daytime and nighttime environments, and collect high-quality, large-scale data such as smoke, cigarette butts, smoking postures, and crowd density. The collected data includes images and videos.

[0053] S1.2: Use the labelimg tool to label the data in the dataset. The labeled elements include the smoker's posture, the glow and smoke of the cigarette butt, the color and shape of the cigarette butt, and the smoking gesture.

[0054] This invention independently creates a large-scale, high-quality dataset and uses labelimg to annotate key recognition targets, greatly improving the quality and diversity of the dataset. The identified targets are highly targeted, providing a good foundation for subsequent recognition accuracy.

[0055] Specifically, S2 is as follows:

[0056] The labeled dataset is divided into three parts: 70% for training, 20% for validation, and 10% for test. It is necessary to ensure that the validation and test sets cover various extreme scenarios, such as dense crowds and low visibility. The training set is stored in the train file in the detection directory, the test set is stored in the test file in the detection directory, and the validation set is stored in the val file in the detection directory.

[0057] Specifically, S3 consists of the following steps:

[0058] S3.1: Configure the Python virtual environment using Anaconda. Open the weight file in VS Code, configure the Python virtual environment configured by Anaconda, execute the command to activate the virtual environment in the VS Code command window, and install the necessary virtual environment libraries.

[0059] S3.2: Dataset optimization strategy. First, the training, validation, and test sets are read. Then, Mosaic data augmentation and HSV color perturbation methods are applied to enhance the images, improving the model's generalization ability and robustness to smoking behavior features. The data augmentation strategy includes brightness adjustment, rotation and translation, scaling, noise addition, random cropping, and color transformation operations on the training data. Brightness adjustment is used to simulate smoking behavior images under different lighting conditions; rotation and translation simulate the posture changes of smokers at different angles and positions; scaling simulates smoking targets captured by surveillance cameras at different distances; noise addition simulates image interference that may occur in actual monitoring scenarios; random cropping is used to simulate partial occlusion or limited field of view; and color transformation is used to simulate visual changes under different weather or lighting conditions.

[0060] This invention introduces various data augmentation techniques into the YOLOv5 smoking behavior detection model, which effectively improves the model's robustness and generalization ability in complex real-world scenarios, and enhances the recognition effect of smoking behavior under different environmental, angle, occlusion and interference conditions.

[0061] S3.3: The optimized training, validation, and test sets are used to train the YOLOv5 visual recognition model. The training results are stored in the runs file. During the training process for smoking behavior detection based on the YOLOv5 visual recognition model, the hyperparameter settings used include: 500 training epochs, batch size of 32, initial learning rate of 0.01, momentum coefficient of 0.937, weight decay coefficient of 0.0005, warmup epochs of 4, color perturbation amplitudes (hsv_h, hsv_s, hsv_v) of 0.015, 0.07, and 0.4 respectively, random angle enhancement (degrees) of 5, translation enhancement ratio of 0.15, random scaling range of 1, slight shear angle of 3, horizontal flip probability of 0.5, and early stopping of 20.

[0062] This invention systematically optimizes the YOLOv5 visual recognition model for smoking behavior detection. By introducing various data augmentation techniques (including color perturbation, rotation, translation, scaling, shearing, and flipping), the robustness and generalization ability of the model under different lighting, viewing angles, distances, and occlusion conditions are significantly improved. Simultaneously, during model training, the training epochs, batch size, initial learning rate, momentum coefficient, and weight decay parameters are appropriately set, and a learning rate warmup mechanism and early stopping strategy are introduced to effectively improve the model's convergence speed and stability, preventing overfitting. The overall method not only achieves a significant improvement in detection accuracy but also has good engineering practicality and deployment adaptability, making it suitable for various public places, key fire prevention areas, and other scenarios with high requirements for smoke and fire safety management.

[0063] like Figure 2As shown in Figures (b) and (c), the distribution of target bounding boxes in the smoking behavior dataset includes the x and y coordinates of the target center point in the image, as well as the width and height distribution of the target. Figures (b) and (c) show that the target center points are mainly concentrated in the central region of the image, indicating that the camera was aimed at the subject during data collection, which is beneficial for the model to learn to focus on the target area. The width and height distribution in Figure (d) shows that most smoking behavior targets are of moderate size, with few extreme samples in the dataset, indicating stable annotation quality. Meanwhile, the heatmap in Figure (a) is densely distributed and without significant skew, reflecting a sufficient and reasonably distributed sample size, ensuring convergence during model training. Overall, this set of labeled data is of high quality and evenly distributed, providing an excellent training foundation for the YOLOv5s model and helping the model achieve better localization and recognition results in the smoking behavior detection task.

[0064] like Figure 3 As shown in the training process indicator graph, the model performs exceptionally well across all metrics. Figure 3 As shown in (a), Box Loss and Objectness Loss continuously decrease with the number of training epochs and eventually converge, indicating that the model has stabilized in object box regression and object confidence judgment, and there is no overfitting phenomenon. The loss on the validation set also decreases significantly, further demonstrating the model's good generalization ability. In terms of performance metrics, Figure 3 In (b), the precision eventually stabilized at around 0.8, and the recall also remained above 0.75, indicating that the model can accurately identify smoking behavior and is not prone to missed detections. Figure 3 The initial loss value of (c)valBox (localization loss) is approximately 0.06, which decreases rapidly during training and eventually converges to approximately 0.038–0.040, indicating a significant improvement in the model's target box localization accuracy and excellent learning performance. The initial loss of valObjectness (confidence loss) is approximately 0.009, which decreases rapidly and eventually stabilizes between 0.004 and 0.005, indicating that the model is becoming increasingly accurate in determining the presence of targets, demonstrating an enhanced ability to identify targets. More importantly, Figure 3 The (d)mAP@0.5 reaches approximately 0.8, and the mAP@0.5:0.95 is close to 0.6, indicating that the model maintains high detection accuracy and localization capability under different IoU thresholds. Therefore, it can be concluded that the YOLOv5s model has significant training effect and excellent detection performance, making it particularly suitable for automatic identification of smoking behavior in real-world scenarios, and possessing strong practicality and deployment value.

[0065] Specifically, S4 consists of the following steps:

[0066] S4.1: Assemble a mobile robot device. The mobile robot device is equipped with a chassis consisting of four Mecanum wheels, a suspension device, and uses an Orbbec camera, SLAMTEC radar, a 7-inch display, a Jetson Nano computing unit (128-core Maxwell GPU and quad-core ARM Cortex-A57 CPU), an STM32 main control unit, large-capacity solid-state storage, a 9-axis IMU sensor, and a 2.4GHz wireless network card.

[0067] S4.2: Deploy the trained model to the mobile robot. After deployment and configuration, store the trained model from the PC into the mobile robot Jetson Nano via a mobile storage device. Use Todesk to remotely connect to the Jetson Nano and store the imported model into a TensorRT inference file for quantization inference. Export the Onnx file and convert it into an FP16 / INT8 TensorRT engine (generating a .wts file and further generating an optimized engine file).

[0068] This invention converts the trained model into ONNX format and deploys it on a Jetson Nano. It then uses the TensorRT toolchain to quantize and accelerate the model, generating a high-efficiency FP16 / INT8 inference engine file. This significantly reduces the computational load and latency of the model on embedded devices, improves the real-time performance and deployment adaptability of smoking behavior detection, and meets the intelligent detection requirements in low-power scenarios.

[0069] S4.3: Real-time detection and inference of smoking behavior are performed using the quantized TensorRT model on the Jetson Nano platform. A camera is used to collect on-site image data, and the deployed YOLOv5-TensorRT model is used for real-time inference detection. The real-time inferred data stream is processed in the Jetson Nano. When a target is detected, the processed signal is transmitted to the microcontroller unit (MCU) via a Universal Asynchronous Receiver / Transmitter (UART), thereby triggering a buzzer alarm.

[0070] like Figure 4 The image shows a comparison of frame rates before and after optimization. In Figure (a), the frame rate of the real-time detection of smoking targets using a camera after training the model with YOLOv5s is shown. As you can see, the FPS is only 6, which does not meet the requirements of high-efficiency real-time acquisition. Therefore, this invention improves the frame rate by optimizing and accelerating the model in step S3.1, which doubles the frame rate while maintaining good detection performance. As shown in Figure (b), the FPS has increased to 13.

[0071] like Figure 5The image shows a comparison of frame rates before and after optimization. During the detection process, the large amount of CPU resources consumed can cause the computing unit to lag or even lose data. As shown in Figure (a), monitoring the CPU resources can be seen to show that the CPU usage is mostly at full capacity. This invention uses TensorRT to call the GPU through step S4.2 to optimize memory management and accelerate the process by calling the GPU. As shown in Figure (b), after optimization, CPU resources are released during the detection process, which greatly optimizes resource management.

[0072] This invention utilizes the Jetson Nano platform to run a quantized TensorRT model, achieving efficient real-time detection of smoking behavior. It acquires on-site images via a camera and performs instant inference and recognition. When smoking behavior is detected, the signal is quickly transmitted to the MCU via UART, triggering a buzzer alarm. It boasts advantages such as fast response speed, reliable alarm mechanism, and suitability for edge device deployment, enhancing the system's practicality and intelligence in real-world scenarios.

[0073] S4.4: After the vision part is completed, the movement part is performed. In this invention, Google's open-source Cartographer algorithm library is used to implement 2D and 3D SLAM (Simultaneous Localization and Mapping). Cartographer achieves efficient and stable map construction by fusing data from multiple sensors and combining map optimization techniques (including multi-threaded backend optimization and problem construction and solving based on Ceres Solver).

[0074] This invention integrates the three core modules of Cartographer: cartographer, cartographer_ros, and ceres-solver, and configures parameters and nodes based on the ROS system. To improve map accuracy, a three-stage SLAM framework of "feature extraction—loop closure detection—backend optimization" is adopted. During the robot's movement control, a wireless handle drives the robot and controls the data release of each sensor; the RViz visualization tool is used in a PC virtual machine to read ROS topic data in real time and complete the mapping task. For loop closure detection, the scan_match strategy is used to correct the position of submaps, effectively mitigating the impact of accumulated errors during long-term mapping, and ultimately generating a high-precision environmental map.

[0075] Furthermore, this invention implements a navigation and obstacle avoidance system based on the ROS navigation package. By configuring the move_base node and integrating the global_planner and local_planner modules, global and local path planning is achieved. The system constructs a global cost map and a local cost map to describe environmental information, and achieves accurate global positioning using a particle filter algorithm through an AMCL (Adaptive Monte Carlo Localization) node. Simultaneously, an extended Kalman filter (EKF) is integrated for multi-sensor data fusion, further improving positioning accuracy and control stability.

[0076] like Figure 6 As shown, users can remotely control navigation points through a host computer. The mobile robot can autonomously plan multiple routes, making detection intelligent. Figure (a) shows a schematic diagram of the mobile robot's perception in a new environment. It can be seen that it clearly establishes a graphic of the current environment and uses red lines to depict the positions of obstacles. Green blocks are used to represent the mobile robot's self-localization. Figure (b) shows that the values ​​1 and 2 are points selected by the user. The mobile robot will patrol in real time according to these points, first going to 1 and then to 2, and then from 2 back to 1, constantly patrolling back and forth. During the patrol, it will autonomously avoid obstacles.

[0077] Through the above systematic integration and debugging, this invention has successfully built a robot platform with autonomous navigation and dynamic obstacle avoidance capabilities. It can operate stably in complex environments and perform high-precision path planning tasks. Moreover, there is no need to worry about computing power. The results calculated on the PC can be interconnected with the mobile robot. It is even possible to use the PC for calculation, while the mobile robot is only responsible for sensor data collection and decision-making.

[0078] In summary, the mobile sentinel robot based on Yolov5s of this invention has the following effects:

[0079] (1) Dynamic recognition and data processing capabilities: Based on the difficulty of implementing the task of banning smoking in public places, this design proposes to use the smoking behavior detection mechanism of YOLOv5s. After high-quality datasets and highly targeted annotations, high-quality training sets, validation sets and test sets for smoking behavior detection are obtained. Then, the images are enhanced by Mosaic data augmentation and HSV color perturbation methods. The training effect of the model is optimized by giving hyperparameters and then deployed on the low-power edge device Jetson nano to realize depth camera image capture and target recognition, which greatly improves the recognition accuracy in multiple scenes. TensorRT is used to infer the model to achieve GPU acceleration. Compared with the traditional recognition frame rate, it is more than doubled. This optimization process does not rely on deep learning frameworks and focuses on GPU acceleration of the inference process. Combined with the low latency and high throughput characteristics of TensorRT, the optimization scheme of this invention improves the performance of target detection and ensures that it can respond quickly and provide accurate detection results in practical applications.

[0080] (2) Detection Range: Traditional detection mainly adopts fixed-point detection, which is inflexible and easily obstructs the field of view. Installing cameras in every corner is too costly, and more images increase the difficulty of supervision. Therefore, this design proposes to use a mobile patrol sentry robot to replace fixed-point camera supervision. Using an STM32 as the control board, it drives four Mecanum wheel motors and is equipped with a suspension device, enabling the robot to move in all directions, greatly increasing the detection range, such as narrow passages, dense crowds, and scenes with many obstacles. Moreover, the device can be expanded with various sensors, such as smoke, temperature, and humidity sensors, so it can capture targets that are difficult to identify visually. The mapping of this navigation patrol uses the Cartographer mapping algorithm, which realizes the effect of multi-sensor fusion data acquisition and closed-loop detection to correct the map position. Compared with traditional mapping, it greatly improves the mapping accuracy of the robot, supports autonomous navigation and cruise. The management personnel only need to mark a few points, and the mobile robot will continuously and autonomously plan its movement trajectory to the target point. The mobile robot will autonomously avoid obstacles that appear along the way according to the algorithm.

[0081] (3) Data transmission: The mobile robot has good data transmission performance. It is equipped with a 2.4GHz network card, which can realize wireless transmission. Alternatively, a transmission method without a network can be selected to directly save the identified target to the solid-state drive. Staff can then view the stored images for easy tracing and analysis.

Claims

1. A method for detecting smoking behavior based on YOLOv5s, characterized in that, Includes the following steps: S1: Collect a dataset of smoking behavior and label the collected dataset using labelimg; S2: Divide the dataset labeled in step S1 to generate training set, validation set and test set; S3: Using a dataset optimization strategy, train the Yolov5s recognition model based on the training set, validation set, and test set generated in step S2; S4: Deploy the trained YOLOv5s recognition model on the robot device, and also deploy TensorRT on the robot device. Collect on-site image data through the camera on the robot, and use the deployed YOLOv5s-TensorRT for real-time inference detection. When smoking behavior is detected, transmit the processing signal to the MCU through UART, thereby triggering the buzzer alarm.

2. The method for detecting smoking behavior based on YOLOv5s according to claim 1, characterized in that, In step S1, the dataset includes images and videos from various environments where smoking behavior needs to be detected.

3. The method for detecting smoking behavior based on YOLOv5s according to claim 1, characterized in that, The elements labeled by labelimg in step S1 include the smoker's posture, the glow and smoke of the cigarette butt, the color and shape of the cigarette butt, and the smoking gesture.

4. The method for detecting smoking behavior based on YOLOv5s according to claim 1, characterized in that, In step S2, the training set accounts for 70%, the validation set accounts for 20%, and the test set accounts for 10%. The training set is stored in the train file in the detection directory, the validation set is stored in the val file in the detection directory, and the test set is stored in the test file in the detection directory.

5. The method for detecting smoking behavior based on YOLOv5s according to claim 1, characterized in that, Step S3 specifically involves: S3.1: Configure a Python virtual environment using Anaconda. Open the weight file in VS Code, configure the Python virtual environment configured by Anaconda, execute the command to activate the virtual environment in the VS Code command window, and install all necessary virtual environment libraries. S3.2: Employing a dataset optimization strategy, data from the training, validation, and test sets are read. Mosaic data augmentation and HSV color perturbation are applied to enhance the images. The Mosaic data augmentation strategy includes brightness adjustment, rotation and translation, scaling, noise addition, random cropping, and color transformation operations on the training data. Brightness adjustment simulates smoking behavior images under different lighting conditions; rotation and translation simulate the posture changes of smokers at different angles and positions; scaling simulates smoking targets captured by surveillance cameras at different distances; noise addition simulates image interference that may occur in actual surveillance scenarios; random cropping simulates partial occlusion or limited field of view; and HSV color perturbation simulates visual changes under different weather or lighting conditions. S3.3: Train the yolov5s visual recognition model using the enhanced training set, validation set, and test set. The training results are stored in the runs file.

6. The method for detecting smoking behavior based on YOLOv5s according to claim 1, characterized in that, In step S3, during training, the training epochs are set to 500, the batch size to 32, the initial learning rate to 0.01, the momentum coefficient to 0.937, the weight decay coefficient to 0.0005, the initial training epochs warmup_epochs to 4, the hue, saturation, and brightness to 0.015, 0.07, and 0.4 respectively, the random angle enhancement to 5, the translation enhancement ratio to 0.15, the random scaling range to 1, the slight clipping angle to 3, the horizontal flip probability to 0.5, and the automatic termination epochs early_stopping to 20.

7. A robot, characterized in that, The method for detecting smoking behavior based on YOLOv5s as described in any one of claims 1-6 above.

8. A robot according to claim 7, characterized in that, The robot incorporates the Cartographer mapping algorithm and the ROS navigation system.

Images