Methods, systems, equipment, and storage media for identifying smoking behavior in industrial parks
By employing a two-stage processing architecture—first human detection and then region clipping—and closed-loop processing within the graphics processor's memory in park monitoring, the accuracy and efficiency issues of small-target smoking behavior recognition in park monitoring have been resolved, achieving efficient and stable smoking behavior recognition.
Patent Information
- Application Number
- CN202610082724.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-21
- Publication Date
- 2026-05-26
AI Technical Summary
Existing methods for identifying smoking behavior face technical challenges in park monitoring scenarios, such as sparse small target detection features, susceptibility to background interference, and sensitivity to occlusion. These challenges result in unstable detection accuracy, significant waste of computational resources, and difficulty in achieving real-time recognition.
It adopts a two-stage processing architecture of first detecting human bodies and then cropping regions. It uses a graphics processing unit (GPU) to perform human target detection and sub-image cropping, and combines transfer learning and closed-loop processing within the GPU memory to improve recognition accuracy and computational efficiency.
It effectively alleviates the problems of sparse features of small targets and background interference, improves the robustness and real-time performance of smoking behavior recognition, reduces computational latency and resource consumption, and achieves the goal of efficient park monitoring.
Smart Images

Figure CN122090370A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of smoking behavior recognition technology, and more specifically, to a method, system, device, and storage medium for smoking behavior recognition in industrial parks. Background Technology
[0002] With the deepening of smart park construction, the demand for public place safety management and civilized behavior supervision is constantly increasing, leading to the widespread application of computer vision-based video surveillance analysis technology. Among these technologies, the automatic identification of smoking behavior in park settings has significant application value in environments such as chemical industrial parks, industrial parks, and enterprise parks, effectively preventing fire risks, ensuring production safety, and maintaining public order. Traditional monitoring methods relying on manual patrols suffer from poor timeliness, limited coverage, and high labor costs, making it difficult to meet the needs of large-scale parks for 24 / 7 uninterrupted monitoring. Therefore, how to achieve accurate, stable, and real-time identification of smoking behavior in park surveillance videos through intelligent technologies has become a pressing technical problem to be solved in the field of park security.
[0003] Existing methods for identifying smoking behavior primarily employ deep learning-based target detection techniques, directly detecting and judging cigarette targets or smoke areas in the original surveillance images. However, park surveillance scenarios are characterized by wide monitoring ranges, high camera mounting positions, and long target distances, resulting in extremely small image sizes for human figures and cigarette targets, typically occupying only a tiny proportion of the total image area. Detecting such small targets faces technical challenges such as sparse feature information, susceptibility to background interference, and sensitivity to occlusion. Direct detection methods struggle to maintain stable detection accuracy in real-world park environments with complex backgrounds, varying lighting conditions, and partial occlusion. Summary of the Invention
[0004] The purpose of this application is to provide a method, system, device, and storage medium for identifying smoking behavior in industrial parks, in order to solve the above-mentioned problems.
[0005] In a first aspect, embodiments of this application provide a method for identifying smoking behavior in a park, the method comprising: acquiring video stream data from park surveillance; decoding the video stream data to obtain image frames; performing human target detection on the image frames to determine the location of each human body in the image frames; cropping sub-images containing the human body from the image frames based on the location of the human body; performing smoking behavior identification on the sub-images to determine whether the human body in the sub-images is smoking; and outputting the smoking behavior identification results of each human body in the image frames.
[0006] In the implementation of the above scheme, a two-stage processing architecture of first human detection and then region cropping is adopted to transform the problem of small target smoking behavior recognition in a large-scale park monitoring scenario into a refined behavior discrimination problem under a normalized scale. This effectively alleviates the contradiction between the sparsity of small target features and background interference, and improves the balance between recognition accuracy and computational efficiency. On the other hand, the step-by-step processing mechanism of decoupling human detection and smoking behavior recognition allows the first stage to quickly filter unmanned areas, avoids dense feature extraction of the entire image, and reduces the ineffective computational overhead of the graphics processor. Furthermore, the sub-image cropping operation based on the human body's location allows the smoking behavior recognition model to focus on the human body and its associated areas, enhancing the model's robustness to the discrimination of smoking behavior under occlusion, viewpoint changes, and long-distance imaging conditions.
[0007] In one implementation of the first aspect, the method further includes: pre-training a smoking recognition model on a publicly available smoking behavior dataset to learn a general feature representation of smoking behavior; wherein the publicly available smoking behavior dataset includes labeled images centered on a human body and in which cigarette butts are visible; fine-tuning the pre-trained smoking recognition model based on park scene labeled data to enable the smoking recognition model to learn the image distribution characteristics under park monitoring scenarios; and using the fine-tuned smoking recognition model to recognize smoking behavior in the sub-images.
[0008] In the implementation of the above scheme, the transfer learning mechanism of pre-training on public datasets and fine-tuning on park scene data enables the smoking recognition model to integrate general smoking behavior features with the specific scene distribution characteristics of the park, which alleviates the problem of decreased model generalization ability caused by differences in park monitoring perspective, lighting, occlusion conditions and public datasets. On the other hand, the transfer training strategy reduces the model's dependence on large-scale park labeled data, and the model parameters can be effectively adjusted with a small amount of park scene data, which improves the model training efficiency and scene adaptation speed.
[0009] In one implementation of the first aspect, the method further includes: calling a graphics processor to execute a video stream data decoding step, a human target detection step, a sub-image cropping step, and a smoking behavior recognition step; wherein the image frame, the sub-image, and the intermediate data generated in the human target detection step and the smoking behavior recognition step are all stored in the video memory of the graphics processor.
[0010] In the implementation of the above scheme, the graphics processor is called to perform the entire process of decoding, detection, cropping and recognition, and the image frames, sub-images and intermediate processing data are stored in the graphics processor's video memory. This eliminates the data copy overhead between the graphics processor and the central processing unit and improves the throughput of the end-to-end processing link. On the other hand, the closed-loop processing mechanism in the graphics processor's video memory avoids the central processing unit's memory bandwidth occupation and reduces processing latency.
[0011] In one implementation of the first aspect, cropping a sub-image containing the human body from the image frame based on the region where the human body is located includes: obtaining a bounding box of the region where the human body is located; expanding the bounding box to obtain an expanded bounding box; and cropping a sub-image from the image frame based on the expanded bounding box; wherein the relative proportion of the human body target in the sub-image is higher than the relative proportion of the human body target in the image frame.
[0012] In the implementation of the above scheme, by acquiring the human bounding box and cropping the sub-image after expansion, the problem of recognizing small targets smoking behavior in a large-scale park monitoring scene is transformed into a refined detection problem under normalized scale. This effectively alleviates the problem of feature sparsity and background interference caused by the long distance of the target and the small imaging size, and improves the model's ability to distinguish cigarette targets. On the other hand, by expanding the operation, while increasing the relative proportion of human targets, the surrounding context region of the human body is preserved, enabling the smoking behavior recognition model to capture the spatial correlation features between the human body and the cigarette, and enhancing the model's adaptability to occlusion and posture changes.
[0013] In one implementation of the first aspect, the method further includes: when there are multiple human bodies in the image frame, performing the cropping operation on the area where each human body is located to obtain a sub-image corresponding to each human body; and performing the smoking behavior recognition step in parallel on the sub-images of each human body to obtain the smoking behavior recognition result corresponding to each sub-image.
[0014] In the implementation of the above scheme, by performing cropping and parallel smoking behavior recognition operations on multiple human bodies in an image frame, parallel processing capability in multi-target scenarios is achieved, improving the recognition efficiency of multiple people smoking in a single frame image and meeting the real-time detection needs of high-density crowds in park monitoring scenarios. On the other hand, by performing cropping and recognition operations independently on the area where each human body is located, feature interference and regional occlusion effects between different human targets are avoided, enabling the smoking behavior recognition model to make fine-grained judgments on individual human body areas and enhancing the recognition accuracy in multi-target scenarios.
[0015] In one implementation of the first aspect, the method further includes: tracking the same human body in multiple consecutive image frames to determine the identity association of the same human body in multiple consecutive image frames; for the same human body, counting the number of frames in which the human body's smoking behavior is identified as having smoking behavior in multiple consecutive image frames; if the number of frames reaches a preset threshold, then determining that the human body has smoking behavior.
[0016] In the implementation of the above scheme, by tracking the same human body in multiple consecutive image frames and counting the number of frames in which smoking behavior is identified as occurring, temporal consistency is introduced into the judgment mechanism. This effectively filters out momentary false detections caused by fluctuations in the quality of a single frame image, environmental interference, or coincidental postures, thus improving the robustness and reliability of smoking behavior judgment. On the other hand, the mechanism of comprehensive judgment of multiple frames makes the alarm triggering conditions match the duration characteristics of the actual smoking behavior, avoiding misjudging brief non-smoking actions such as hand approaching the mouth as smoking behavior and reducing the false alarm rate.
[0017] In one implementation of the first aspect, after determining that a person in the sub-image is smoking, the method further includes: locating a face region in the sub-image and extracting facial feature information from the face region; comparing the extracted facial feature information with a preset face database; if a match is found in the preset face database, obtaining the person's identity information; if a match fails in the preset face database, assigning a unique identifier to the person; and associating the person's identity information or the unique identifier with the smoking behavior recognition result and outputting the result.
[0018] In the implementation of the above scheme, by further locating the facial region and extracting feature information for identity comparison after identifying smoking behavior, the correlation output of behavior detection and identity recognition is realized, so that alarm events can be accurately matched with specific personnel, which is convenient for park management to trace behavior and confirm responsibility. On the other hand, the facial feature comparison mechanism can distinguish between registered personnel and temporary visitors in the preset facial database. For those who are matched successfully, the identity information is directly obtained, and for those who are not matched, a unique identifier is automatically assigned, which takes into account both the management efficiency of registered personnel and the complete recording of all detection targets.
[0019] In one implementation of the first aspect, the method further includes: generating a structured alarm event and reporting the structured alarm event to a management system; wherein the structured alarm event includes either the personnel identity information or the unique identifier, as well as the smoking behavior identification result and timestamp information; the reporting method of the structured alarm event includes local storage, network push, or message notification; triggering a linkage response of related systems based on the structured alarm event; wherein the linkage response includes access control operation, voice broadcast, or message push reminder.
[0020] In the implementation of the above solution, structured alarm events containing personnel identification, smoking behavior recognition results, and timestamps are generated and reported via local storage, network push, or message notification. This achieves standardized encapsulation and multi-channel transmission of alarm information, ensuring that the management system can receive alarms in a timely and reliable manner. On the other hand, based on the linkage response mechanism triggered by structured alarm events to access control, voice broadcast, or message push, the smoking behavior recognition results are transformed into actual park security control actions, forming a closed-loop management of detection, alarm, and response, and improving the automated handling capabilities of park security management.
[0021] Secondly, embodiments of this application provide a smoking behavior recognition system for industrial parks, including a graphics processor and an image acquisition module communicatively connected to the graphics processor, wherein: The image acquisition module is used to acquire video stream data from the park's surveillance system and send the video stream data to the graphics processor. The graphics processor is configured to: acquire the video stream data; decode the video stream data to obtain image frames; perform human target detection on the image frames to determine the location of each human body in the image frames; based on the location of the human body, crop out a sub-image containing the human body from the image frames; perform smoking behavior recognition on the sub-images to determine whether the human body in the sub-images is smoking; and output the smoking behavior recognition results of each human body in the image frames.
[0022] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other through the communication bus; the memory stores computer program instructions that can be executed by the processor, and the computer program instructions are read and executed by the processor to perform the method provided in the first aspect or any possible implementation of the first aspect.
[0023] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the method provided in the first aspect or any possible implementation thereof.
[0024] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the method provided by the first aspect or any possible implementation of the first aspect.
[0025] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims and drawings. Attached Figure Description
[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 Examples of images from a public dataset provided in this application's embodiments; Figure 2 This is an example of an image from the park monitoring dataset provided in an embodiment of this application; Figure 3 A flowchart illustrating the smoking behavior recognition method for industrial parks provided in this application embodiment; Figure 4 Provided for the embodiments of this application Figure 2 A schematic diagram of the cropped sub-image; Figure 5 This is a schematic diagram of the visualization rendering result in a certain application scenario provided in an embodiment of this application; Figure 6 Provided for the embodiments of this application Figure 5 Enlarged view of the detection frame in the middle; Figure 7 A schematic diagram of the structure of a smoking behavior recognition device for a park provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0028] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.
[0029] As a typical long-distance, wide-field-of-view video surveillance environment, park surveillance scenarios often involve cameras mounted at high positions in actual engineering deployments, covering a large horizontal field of view and extending the monitoring radius to considerable distances. In this configuration, the size of adult human figures in the video image is typically small, while the image size of cigarette targets is even smaller. This makes it difficult for the feature extraction layer in traditional target detection models to capture sufficient semantic information. Even with multi-scale feature fusion strategies, shallow features, while retaining positional information, lack semantic discriminative power, and deep features, while possessing semantic abstraction capabilities, lose fine-grained localization information due to repeated pooling operations, still resulting in an imbalance between semantics and recognition results.
[0030] The mainstream implementation of existing smoking behavior recognition technology adopts a single-stage end-to-end detection architecture. A typical approach involves a detection model based on the YOLO or Faster R-CNN framework, which performs dense anchor box scanning across the entire image to classify cigarette targets and regress bounding boxes in candidate regions. This type of approach relies on large-scale labeled datasets for model training, with data annotation focusing on accurate cigarette target selection. Model optimization aims to improve the precision and recall of cigarette detection. During the inference phase, association matching is performed based on the Intersection-over-Union (IoU) threshold between the cigarette detection bounding box and a preset human body region. When the IoU exceeds the preset threshold, the region is identified as a human body associated with a cigarette, triggering a smoking behavior alarm.
[0031] However, the above-mentioned solution has the following drawbacks when applied in a park setting: (1) There are significant domain differences between the park monitoring dataset and the publicly available dataset, such as... Figure 1 and Figure 2 As shown, cigarette targets in public datasets are usually clear, have little occlusion, and simple backgrounds. However, park scenes have problems such as dynamic lighting, complex background interference, mutual occlusion of people, and smoke blurring. When the model is deployed across scenes, its generalization ability drops sharply, and the false detection rate is generally high, which cannot meet the reliability requirements of park security applications. (2) The full-image dense detection mechanism leads to a waste of computing resources. For uninhabited areas or confirmed non-smokers, feature extraction is still repeatedly performed, which makes it difficult to achieve real-time processing speed on low-power edge computing devices. (3) Small target detection is sensitive to the resolution of the input image. In order to improve the visibility of cigarette butts, a high input resolution needs to be maintained, which further aggravates the computing load and forms a contradiction between accuracy and speed that is difficult to reconcile.
[0032] In view of this, this application provides a smoking behavior recognition method for parks. This method transforms the problem of recognizing small-target smoking behavior in a large-scale park monitoring scene into a refined behavior discrimination problem under a normalized scale through a two-stage processing architecture of first human detection and then region cropping. This effectively alleviates the contradiction between the sparsity of small-target features and background interference, and improves the balance between recognition accuracy and computational efficiency. On the other hand, the step-by-step processing mechanism of decoupling human detection and smoking behavior recognition allows the first stage to quickly filter unoccupied areas, avoiding dense feature extraction of the entire image and reducing the ineffective computational overhead of the graphics processor. Furthermore, the sub-image cropping operation based on the human body's location allows the smoking behavior recognition model to focus on the human body and its associated areas, enhancing the model's robustness to the discrimination of smoking behavior under occlusion, viewpoint changes, and long-distance imaging conditions.
[0033] Please see Figure 3 The illustrated flowchart illustrates a smoking behavior recognition method for industrial parks provided in this application embodiment. This smoking behavior recognition method for industrial parks can be applied to electronic devices, which may include physical devices such as servers, PCs, tablets, or smartphones, or virtual devices such as virtual machines or containers. The electronic device can be a single device, a combination of multiple devices, or a cluster of a large number of devices. The aforementioned smoking behavior recognition method for industrial parks may include: Step S110: Obtain video stream data from the park's surveillance system.
[0034] The aforementioned video stream data refers to a sequence of digital video data that is continuously transmitted after being compressed and encoded using video coding standards (such as H.264 / AVC, H.265 / HEVC, etc.) and according to real-time streaming protocols (such as RTSP (Real Time Streaming Protocol), RTMP (Real-Time Messaging Protocol), or ONVIF (Open Network Video Interface Forum) standard protocols. This video data sequence consists of consecutive video frames, each containing encoded image data, timestamp information, and possibly metadata. Its data format follows the specifications established by the International Organization for Standardization (ISO) and the Moving Pictures Experts Group (MPEG), possessing temporal continuity and streaming media characteristics, and is typically transmitted in the form of network data packets.
[0035] The video stream data of the park monitoring can be obtained in step S110 above through the following methods: network cameras (IP cameras) deployed in the park push real-time video streams through network interfaces using the RTSP protocol. The back-end processing system establishes a transport layer connection through standard network sockets, receives RTP data packets and reassembles them into a complete video stream after protocol handshake and media negotiation; or it can be obtained by pulling the stream through the standard API interface provided by the video management platform. This platform acts as middleware to centrally manage all camera resources in the park and supports calling video stream data of specified channels on demand; or it can be obtained by playback through local video storage devices (such as NVR or NAS), reading the stored video data blocks based on timestamp positioning or file index and parsing them into a video stream sequence.
[0036] Step S120: Decode the video stream data to obtain image frames.
[0037] The above-mentioned step S120, which decodes the video stream data, may include parsing the Video Coding Layer (VCL) syntax structure and performing steps such as entropy decoding, inverse quantization, inverse transform, and motion compensation reconstruction. One implementation of this decoding method is as follows: extracting Network Abstraction Layer (NAL) units from the video stream, and identifying the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and Slice data based on the type identifier in the NAL unit header. The process involves several steps: First, initializing the video sequence and configuring the decoding environment. Then, based on the context-adaptive binary arithmetic coding or context-adaptive variable-length coding algorithm defined by the H.264 / H.265 coding standard, entropy decoding is performed on the bitstream to obtain quantization coefficients, motion vectors, and prediction mode information. Second, the frequency domain coefficients are restored to residual image blocks through inverse quantization and inverse transform operations. Third, the predicted image blocks are reconstructed by combining intra-frame prediction or inter-frame prediction mechanisms based on motion compensation. Finally, the residual and prediction results are superimposed to generate the decoded image. The decoded image can be stored in the YUV color space (such as YUV420 or NV12 format) and can be converted to RGB or BGR format as needed through a color space conversion matrix to form image frame data that can be directly used by subsequent schemes.
[0038] The video stream decoding process described above can be accelerated using hardware acceleration techniques. For example, the computational load on the central processing unit (CPU) can be offloaded by the hardware decoding engine integrated in the graphics processing unit (GPU) or dedicated video processing unit (VPU). The decoding task can be executed by the GPU's dedicated video decoding unit (such as NVIDIA's NVDEC, AMD's UVD, Intel's Quick Sync Video, etc.). These hardware units have built-in fixed-function circuits optimized for the H.264 / H.265 encoding standard, supporting parallel processing of entropy decoding, inverse quantization, inverse transform, and motion compensation operations for multiple macroblocks or coding tree units. At the implementation level, the compressed bitstream can be directly submitted to the hardware decoding unit by calling the application programming interfaces provided by the vendor (such as NVIDIA's CUDA Video Decoder API, Intel's Media SDK, or the cross-platform Video Acceleration API). The decoded image frames can be stored in the GPU's video memory in a zero-copy manner for subsequent computing units to perform object detection and behavior recognition, thereby avoiding the data transfer overhead between the CPU and the GPU.
[0039] Step S130: Perform human target detection on the image frame to determine the location of each human body in the image frame.
[0040] The core purpose of step S130 above is to perform human target presence detection in the image frame. If the human detection result is empty, it indicates that there is no human target in the current image frame. Therefore, the subsequent processing flow for that frame is immediately terminated, and the process jumps directly to the acquisition and decoding of the next frame. This avoids performing redundant sub-image cropping and smoking behavior recognition calculations on unoccupied image frames, reducing the overall computational load and unnecessary energy consumption. If the human detection result includes a target instance, the bounding box of the human target in the original image is output.
[0041] The aforementioned human target detection can be achieved using a human detection model deployed on a graphics processor. The human detection model can be implemented using models such as the YOLO (You Only Look Once) series or the SSD (Single Shot MultiBox Detector) framework. Step S130 described above implements one method for human target detection in image frames. For example: the image frame is preprocessed to a fixed size (e.g., 640×640 or 1280×720 pixels) required for model input, and pixel values are normalized. Then, multi-scale feature maps are generated through the multi-layer convolutional feature extraction structure of the backbone network, and semantic and positional information at different levels are fused using Feature Pyramid Networks (FPN) or Path Aggregation Networks (PAN). The detection head presets multiple anchor boxes at each spatial location, and performs classification and regression branch calculations in parallel for each anchor box. The classification branch outputs a confidence score for the human category using the Softmax or Sigmoid function, and the regression branch predicts the positional offset of the anchor box relative to the real human boundary. The preliminary detection results output by the model are filtered by a post-processing algorithm, using Non-Maximum Suppression (NMS). The suppression mechanism removes redundant detection boxes with high overlap and low confidence, and retains candidate boxes with confidence scores exceeding a preset threshold (such as 0.5) as the final human detection results. Each detection result is represented in the form of a bounding box, which includes the coordinates of the upper left and lower right corners or the center point of the rectangle plus the width and height parameters, thereby determining the precise location of each human body in the image frame.
[0042] It's understandable that in park surveillance scenarios, image frames typically cover a vast spatial area, resulting in a low pixel footprint for cigarette targets. Directly performing smoking behavior recognition on the entire image yields poor results. Unfiltered full-image processing forces the recognition model to extract redundant features from massive background areas at extremely high resolution, leading to inefficient consumption of computational resources and difficulty in constructing stable discrimination boundaries due to the sparse feature information of small targets. The proposed solution, however, employs a strategy of first detecting human targets. By locating human bounding boxes, the recognition range is constrained to a spatial neighborhood strongly correlated with smoking behavior, transforming the original large-scale image frame into a local sub-image centered on the human body, thus normalizing the scale of the detected target. This cascaded architecture allows subsequent smoking recognition models to break free from dependence on extremely small target features, fully capturing the visual semantic features of the human-cigarette interaction area at a fixed and reasonable input resolution. Simultaneously, it eliminates the computational complexity of full-image scanning, providing architectural guarantees for real-time processing. In addition, human detection, as a pre-constraint, can eliminate false detections of cigarette-like objects that are not related to human bodies from a spatial perspective, reducing the search space for the recognition task and thus lowering the probability of false alarms caused by background interference.
[0043] Step S140: Based on the location of the human body, crop out a sub-image containing the human body from the image frame.
[0044] Optionally, step S140 may include: obtaining the bounding box of the region where the human body is located; expanding the bounding box to obtain an expanded bounding box; cropping a sub-image from the image frame based on the expanded bounding box; wherein the relative proportion of the human target in the sub-image is higher than the relative proportion of the human target in the image frame.
[0045] It is understandable that the original human detection bounding box is used to define the range of the human body contour. Under long-distance imaging conditions, the cigarette target is often located near the boundary of the human body bounding box or partially extends outside the box. The above-mentioned expansion operation, by enlarging the width and height of the bounding box, introduces the surrounding background area while maintaining the center position of the human body. This not only increases the relative proportion of the human target in the sub-image, making its imaging scale closer to the distribution characteristics of the public dataset, but also preserves complete action context information, providing sufficient visual cues for the smoking recognition model to determine hand posture, cigarette position, and the spatial relationship between the two. In addition, the above-mentioned expansion and cropping mechanism can compensate for the detection box positioning deviation caused by changes in human posture (such as turning to the side or raising the hand), avoiding the omission of cigarette targets that exceed the human body contour due to overly tight selection, and enhancing the completeness of cropping coverage for smoking actions with multiple postures and angles.
[0046] When setting the expansion ratio, the following factors can be considered: (1) The relative proportion of human targets in the original image frame and the spatial distribution characteristics of cigarette targets: The human imaging size in the park monitoring scene is affected by the camera mounting height, focal length parameters and target distance. When the proportion of human targets is small, the expansion factor needs to be increased appropriately to ensure that the cigarette targets have sufficient pixel resolution in the cropped sub-image for subsequent recognition models to extract features; conversely, if the proportion of human targets is large, the expansion factor should be reduced accordingly to avoid introducing too much redundant background. At the same time, the spatial position of cigarette targets relative to human targets shows a statistical distribution pattern, usually concentrated in the interactive area formed by the hands and face. The center of this area is deviated from the geometric center of the human body frame. The setting of the expansion factor needs to ensure that the interactive area falls completely into the cropping range without being truncated. Usually, the safe expansion boundary needs to be determined by analyzing the distance distribution statistics between cigarette targets and human body frame boundaries in the training dataset. (2) Dynamic range and amplitude of human posture: Smoking behavior involves a continuous movement of the hand from a naturally hanging position to the face area. During this process, the hand trajectory may exceed the coverage of the static human body detection box, especially in postures such as turning to the side, turning around, or raising the arm, the hand position may extend outside the human body box. The expansion factor needs to be set according to the maximum offset of the key points of the hand relative to the human body box in the smoking action sequence to ensure that the cigarette and the hand can be completely included in the expanded bounding box within the typical posture change range, avoiding the problem of region omission due to posture diversity. (3) Balance between the need to preserve background context information and interference suppression: Moderate expansion can introduce the environmental context around the human body, providing auxiliary discrimination clues for the smoking behavior recognition model, such as the starting position of the hand movement trajectory, the distribution of surrounding people, and scene type information, which helps to distinguish real smoking behavior from similar actions (such as drinking water, making a phone call). However, excessive expansion will introduce too much irrelevant background, increase the risk of false detection of smoke-like objects (such as pens, tools) in the background, and lead to a decrease in sub-image resolution. Therefore, the upper limit of the expansion factor should be constrained by not reducing the relative proportion of human targets. (4) Computational resources and model input resolution limitations: A larger expansion factor leads to an increase in the size of the cropped sub-image. If it exceeds the maximum input resolution of the smoking behavior recognition model, scaling is required. This process may introduce interpolation blur and change the original target size distribution. Therefore, the setting of the expansion factor needs to be calculated in combination with the tensor dimension of the model input layer, the graphics processor memory capacity, and the original resolution of the target detection stage output to ensure that the cropped sub-image can be directly input into the recognition model without scaling or with only minimal scaling, thus maintaining a balance between computational efficiency and feature fidelity.
[0047] In a certain application scenario, the cropped sub-image is as follows: Figure 4As shown, it can be seen that the relative proportion of the human body in the image area is increased in the sub-image, and the pixel resolution of the cigarette butt region is increased accordingly, making the visual features of the cigarette clearer and more identifiable. In addition, the background complexity in the sub-image is also effectively suppressed, and interfering elements such as trees and lawns are effectively eliminated. The compositional features of the sub-image are close to the human-centered distribution pattern in the public dataset, providing the smoking behavior recognition model with an input data distribution that is more consistent with the training phase.
[0048] The above-mentioned scheme transforms the problem of recognizing small targets smoking behavior in a large-scale park monitoring scenario into a refined detection problem at a normalized scale by acquiring the human bounding box and cropping the sub-image after wide expansion. This effectively alleviates the problems of feature sparsity and background interference caused by the long distance of the target and the small image size, and improves the model's ability to distinguish cigarette targets. On the other hand, by expanding the bounding box, the relative proportion of the human target is increased while the surrounding context region of the human body is preserved, enabling the smoking behavior recognition model to capture the spatial correlation features between the human body and the cigarette, and enhancing the model's adaptability to occlusion and pose changes.
[0049] Step S150: Perform smoking behavior recognition on the sub-image to determine whether the human body in the sub-image is smoking.
[0050] The aforementioned smoking behavior recognition can be achieved using a smoking recognition model (such as the YOLOv8n model) deployed in a graphics processor. The sub-image, after preprocessing (including scaling, pixel value normalization, and data type conversion), is input into the backbone network of the smoking recognition model. Multi-scale feature maps are extracted through multi-layer convolution, pooling, and activation functions. The smoking recognition model can pre-set anchor boxes at each spatial location, and compute classification branches in parallel to output the confidence score of each anchor box corresponding to the smoking behavior category. Simultaneously, the regression branch predicts the coordinate offset of the anchor box relative to the actual behavior area. The smoking recognition model can directly output image-level binary classification probabilities, calculating the confidence score of the presence of smoking behavior using the Softmax or Sigmoid function. The initial results output by the model are filtered by a post-processing mechanism. Low-scoring candidate boxes or low-probability samples are filtered using a pre-set confidence threshold, and overlapping detection boxes are eliminated using a non-maximum suppression algorithm. The final retained detection results are marked with bounding boxes indicating the location of the cigarette or hand-cigarette interaction area, while the classification results represent the presence of behavior as probability values, thus enabling the determination of whether a person in the sub-image is smoking.
[0051] Optionally, the above-mentioned method for identifying smoking behavior in industrial parks may further include: pre-training a smoking recognition model on a publicly available smoking behavior dataset to learn a general feature representation of smoking behavior; wherein the publicly available smoking behavior dataset includes labeled images centered on the human body and in which cigarette butts are visible; fine-tuning the pre-trained smoking recognition model based on labeled data of industrial park scenes to enable the smoking recognition model to learn the image distribution characteristics under industrial park monitoring scenes; and using the fine-tuned smoking recognition model to identify smoking behavior in sub-images.
[0052] The above-mentioned park scene annotation data can be collected and constructed from multiple sources to form a training sample set covering diverse imaging conditions. The main sources of park scene annotation data include: (1) real-time video stream data collected by network cameras deployed at various monitoring points in the park, which are manually inspected to select segments containing suspected smoking behavior and extract images frame by frame; (2) historical video data, which are retrieved using timestamp indexes for key time periods (such as peak hours in rest areas and records of areas with high incidence of violations) and exported in batches; (3) simulated datasets, which are organized by volunteers to demonstrate smoking actions in the actual environment of the park, covering different distances, angles, lighting and occlusion scenarios. After collection, the original images are accurately annotated by professional annotators using polygon selection or rectangular box annotation methods to accurately annotate the cigarette targets and human body boundaries, generating annotation files that conform to the YOLO format standard, including category index, normalized center coordinates, width and height parameters.
[0053] The transfer learning mechanism in the above scheme achieves knowledge transfer from general features to scene-specific features through a two-stage training strategy, wherein: Phase 1: Pre-training the basic detection model on a publicly available smoking behavior dataset. This dataset consists of images centered on the human body with clearly visible cigarette butts. The annotation information covers the cigarette position and smoking action category, enabling the model to learn the general visual semantic features of smoking behavior through a large number of samples, including cross-scene invariant features such as cigarette shape, hand gesture, and spatial relationship between hand and face, forming pre-trained weights with basic discrimination capabilities. The second stage involves fine-tuning the pre-trained model based on labeled data from the park scene. This park scene data originates from video streams collected by actual deployed surveillance equipment. After labeling, smoking behavior samples that conform to the imaging characteristics of the park are generated. These samples are characterized by long target distances, small image sizes, complex backgrounds, and frequent occlusions. The fine-tuning process uses a low learning rate to locally adjust the model parameters, enabling the smoking recognition model to learn the image distribution characteristics specific to the park scene while retaining the general feature representations of the pre-trained model. These characteristics include cigarette butt color shifts under specific lighting conditions, long-distance blurring features, and dynamic background interference patterns, achieving feature adaptation from the general domain to the target domain.
[0054] In the first stage, the backbone network parameters of the smoking recognition model can be fixed, and only the weights of the detection head or classification head are updated. The training cycle is controlled within 10 to 20 iterations. This allows the smoking recognition model to quickly adapt to the target category distribution and annotation format of the park scene while retaining the pre-trained general feature representation. This avoids excessive deviation of the backbone network parameters due to an excessively high learning rate in the early stage, which would destroy the learned general visual semantics. In the second stage, all network layer parameters of the smoking recognition model can be unfrozen, and fine-tuning training can continue with a lower learning rate. This allows the backbone network to slowly learn the image distribution characteristics unique to the park scene, including the blurred features of small targets at a distance, complex background interference patterns, and color shifts under specific lighting conditions. At the same time, the low learning rate constraint prevents the model from overfitting on limited scene data, ensuring that the fine-tuned model retains its cross-scene generalization ability while improving recognition accuracy in the park monitoring environment.
[0055] Furthermore, to improve the generalization performance and robustness of the smoking recognition model, multi-dimensional data augmentation processing can be applied to the training data. For example, input images can undergo spatial geometric transformations, including translation to simulate camera position shifts, rotation to cover different human orientations, horizontal or vertical flipping to enhance pose diversity, and shearing to produce viewpoint distortion effects. Color space perturbations can also be applied to the training images, including hue shifts to adapt to different lighting conditions, saturation adjustments to address color depth variations, brightness transformations to simulate day and night lighting differences, and contrast adjustments to enhance the dynamic range of light and dark areas. These augmentation operations can be randomly combined and executed online during training, exposing the model to diverse image distribution samples during the training phase. This effectively suppresses overfitting to specific imaging conditions and enhances the model's adaptability to dynamic factors in park monitoring scenarios (such as personnel movement, slight camera shake, and ambient light fluctuations), thereby improving the recognition stability and generalization ability of the smoking recognition model in actual deployment environments.
[0056] The training data can be divided into three subsets: training set, validation set, and test set. The division of the training and validation sets follows a predetermined ratio, for example, the training set accounts for 80% of the total labeled data, and the validation set accounts for 20%. The partitioning process can be performed using random sampling or stratified sampling strategies to ensure that the sample distribution in each subset maintains statistical consistency with the original dataset. The training set is used to perform gradient updates and backpropagation optimization of model parameters, driving the model to learn the discriminative features of smoking behavior. The validation set is independent of the training process and is used to monitor changes in the model's generalization performance during training. It supports hyperparameter tuning, model architecture selection, and early stopping mechanism judgment to prevent overfitting on the training set. The test set is a separate, additional dataset that is not used during model training and validation. Its samples are collected from park surveillance videos but excluded from the training labeled data. It is specifically used for final performance evaluation after model training to objectively measure the model's accuracy, recall, and false positive rate on unknown data, reflecting the model's true generalization ability and engineering deployment effectiveness.
[0057] In the above scheme, the transfer learning mechanism of pre-training on public datasets and fine-tuning on park scene data enables the smoking recognition model to integrate general smoking behavior features with the specific scene distribution characteristics of the park, which alleviates the problem of decreased model generalization ability caused by differences in park monitoring perspective, lighting, occlusion conditions and public datasets. On the other hand, the transfer training strategy reduces the model's dependence on large-scale park labeled data, and the model parameters can be effectively adjusted with a small amount of park scene data, which improves the model training efficiency and scene adaptation speed.
[0058] Optionally, the above-mentioned smoking behavior recognition method for parks may further include: calling the graphics processor to execute video stream data decoding steps, human target detection steps, sub-image cropping steps, and smoking behavior recognition steps; wherein, the image frames, sub-images, and intermediate data generated in the human target detection steps and smoking behavior recognition steps are all stored in the graphics processor's video memory.
[0059] The aforementioned Graphics Processing Unit (GPU) is a specialized integrated circuit designed specifically for massively parallel computing tasks. Its architecture consists of thousands of streaming multiprocessors, each containing dozens to hundreds of CUDA cores (or similar SIMD execution units), supporting a Single Instruction Multithreaded (SIMT) execution model. GPUs employ a many-core design philosophy, primarily using transistor resources to deploy arithmetic logic units. Their cache hierarchy is relatively simplified, and they are equipped with high-bandwidth video memory to support large-scale data throughput. GPUs achieve thread-level parallelism through a Warp scheduler, enabling hundreds or thousands of threads to perform the same operation within the same clock cycle. Their parallel computing capabilities are suitable for vectorized computationally intensive tasks such as matrix multiplication and convolution operations. In the field of deep learning, GPU architecture can significantly accelerate the forward and backward propagation processes of neural networks. By residing weight parameters, activation values, and gradient data in video memory and utilizing parallel computing frameworks such as CUDA or OpenCL to achieve high parallelism in inter-layer computation, training and inference times can be reduced by several orders of magnitude. The graphics processing unit also integrates dedicated hardware units, including a texture mapping unit, a rasterization engine, and a Tensor Core. The latter supports mixed-precision matrix multiplication and addition operations, further improving the computational efficiency of deep learning tasks. The video memory uses GDDR or HBM technology, providing bandwidth of hundreds to thousands of gigabytes per second to meet the data access speed requirements of parallel computing. Simultaneously, the memory controller enables independent access to multiple partitions, reducing memory access conflicts.
[0060] In the above scheme, after the video stream data is received via the network interface, it is directly submitted to the graphics processor hardware decoding unit. This unit performs entropy decoding, inverse quantization, and motion compensation operations in parallel using a dedicated decoding circuit. The generated image frames are stored in video memory in device memory format, without being sent back to the central processing unit (CPU) memory. Subsequent steps such as human target detection, sub-image cropping, and smoking behavior recognition are all executed by the graphics processor computing unit. The weight parameters of the human detection model and the smoking recognition model are pre-loaded into the constant memory area of the video memory. The convolution operation in the feature extraction process is accelerated in parallel by CUDA cores or tensor cores, and the activation values and intermediate feature maps are allocated and stored in the dynamic memory area of the video memory. The sub-image cropping operation is achieved through video memory address offset and data pointer relocation, without the need for physical data relocation. After post-processing operations such as non-maximum suppression, the recognition results are directly generated by the graphics processor into structured event data and mapped from video memory to user-space memory for the reporting module to read. This mechanism eliminates bidirectional data copying (Host-to-Device and Device-to-Host transmission) between the central processing unit and the graphics processing unit in traditional heterogeneous computing architectures, avoids PCIe bus bandwidth limitations and transmission delay overhead, reduces the data transmission time of the complete processing link, thereby improving the utilization of graphics processing unit computing resources, increasing overall throughput, and maintaining the pipeline parallel characteristics of data processing to meet the real-time requirements of concurrent processing of multiple video streams.
[0061] The above solution calls the graphics processor to perform the entire process of decoding, detection, cropping and recognition, and stores the image frames, sub-images and intermediate processing data in the graphics processor's video memory, eliminating the data copy overhead between the graphics processor and the central processing unit and improving the throughput of the end-to-end processing link; on the other hand, the closed-loop processing mechanism in the graphics processor's video memory avoids the central processing unit's memory bandwidth occupation and reduces processing latency.
[0062] Optionally, the above-mentioned smoking behavior recognition method for parks may further include: when there are multiple human bodies in an image frame, performing a cropping operation on the area where each human body is located to obtain a sub-image corresponding to each human body; performing smoking behavior recognition steps in parallel on the sub-images of each human body to obtain the smoking behavior recognition result corresponding to each sub-image.
[0063] It is understandable that when multiple human bodies exist in an image frame, the multi-human parallel detection and recognition mechanism provided by the above scheme can be adopted. This mechanism is implemented by decoupling the detection and recognition of multiple human targets in a single frame image into independent parallel processing units. Specifically, after the image frame outputs multiple human bounding boxes through the human target detection step, the processing flow treats the region where each human body is located as an independent region of interest, and performs bounding box expansion and cropping operations respectively to generate a corresponding number of sub-image sets. The cropping operation can be scheduled in parallel based on the multi-stream or multi-instance technology of the graphics processor. The expansion calculation and memory address allocation of each ROI are executed asynchronously by independent CUDA streams. The cropped sub-images are stored continuously or in blocks in the graphics memory to form a batch input tensor. The smoking behavior recognition model adopts the batch processing mode, combining the sub-images corresponding to each human body into a single batch and submitting them to the graphics processor computing unit. It utilizes the parallel computing characteristics within the model to perform forward inference simultaneously. Each sub-image maintains an independent feature extraction and classification regression path within the batch, sharing model weights but generating separate activation values and prediction results. In the output stage of the recognition results, the batch prediction tensor is decoupled into confidence scores and bounding box parameters corresponding to each individual through batch dimension splitting operation, and finally outputs the independent smoking behavior recognition results for each individual.
[0064] The above-mentioned scheme achieves parallel processing capabilities in multi-target scenarios by performing cropping and parallel smoking behavior recognition operations on multiple human bodies in an image frame, thereby improving the recognition efficiency of multiple people smoking in a single frame image and meeting the real-time detection needs of high-density crowds in park monitoring scenarios. On the other hand, by performing cropping and recognition operations independently on the area where each human body is located, the feature interference and regional occlusion effects between different human targets are avoided, enabling the smoking behavior recognition model to make fine-grained judgments on individual human body areas and enhancing the recognition accuracy in multi-target scenarios.
[0065] Step S160: Output the smoking behavior recognition results of each person in the image frame.
[0066] Optionally, the above-mentioned smoking behavior recognition method for parks may further include: tracking the same human body in multiple consecutive image frames to determine the identity association of the same human body in multiple consecutive image frames; for the same human body, counting the number of frames in which the smoking behavior recognition result of the human body in multiple consecutive image frames is that smoking behavior exists; if the number of frames reaches a preset number threshold, it is determined that the human body has smoking behavior.
[0067] The above scheme can track the same human body across multiple image frames using a multi-target tracking algorithm. For the bounding boxes of each human body detected in the current image frame, the multi-target tracking algorithm can perform inter-frame matching based on a motion model and an appearance model. The motion model uses a Kalman filter to predict the position distribution of each human body in the next frame, generating predicted bounding boxes; the appearance model extracts the appearance feature vectors of each human body region through a deep convolutional neural network, constructing a temporal feature cache queue. In the data association stage, the intersection-union distance matrix between the detected boxes and predicted boxes in the current frame is calculated, and the cosine similarity matrix between the appearance features of the detected boxes in the current frame and the historical feature queue is also calculated. The optimal allocation is solved using the Hungarian algorithm or a greedy matching strategy, establishing a correspondence between each detected box and an existing trajectory. For successfully matched detected boxes, the identity identifier of the previous frame is inherited; for unmatched detected boxes, a new trajectory is initialized and a unique identity identifier is assigned; for existing trajectories that are unmatched for multiple consecutive frames, a trajectory deletion operation is performed. This mechanism provides a temporal consistency basis for the continuous frame count statistics of smoking behavior by maintaining a stable human body identity chain across consecutive image frames.
[0068] When setting the above preset threshold, the following factors can be considered: (1) Duration characteristics of smoking behavior: The actual smoking action usually lasts for a certain period of time from raising the hand to establishing the smoking posture. Combined with the frame rate of the monitoring video, it can be converted into the corresponding number of frames. This frame range constitutes the benchmark of the lower limit of the threshold. If the threshold is too low, it will trigger false alarms due to the short-term non-smoking action of the hand approaching the mouth. If the threshold is too high, it may lead to the missed detection of rapid smoking action. (2) Video stream frame rate: The frame rate directly affects the number of frames per unit time. Under a higher frame rate, the threshold should be increased accordingly to maintain the same physical time span. (3) False alarm characteristics of the identification model: The threshold needs to be higher than the number of occasional positive frames caused by model false detection. By analyzing the single frame false alarm rate distribution on the validation set, it is ensured that the threshold can effectively filter random false alarms, while being lower than the number of interrupted frames caused by the actual smoking behavior due to occlusion or posture change, thus balancing the risk of missed detection. (4) Real-time alarm sensitivity requirements: The threshold setting needs to take into account the real-time response requirements of the park management scenario. Too low a threshold may lead to frequent false alarms and reduce credibility, while too high a threshold will delay the real alarm and affect the timeliness of handling.
[0069] The above scheme tracks the same human body across multiple consecutive image frames and counts the number of frames where smoking behavior is identified as occurring. By introducing temporal consistency into the judgment mechanism, it effectively filters out momentary false detections caused by fluctuations in the quality of a single frame image, environmental interference, or coincidental poses, thus improving the robustness and reliability of smoking behavior judgment. On the other hand, the mechanism of comprehensive judgment across multiple frames ensures that the alarm triggering conditions match the duration characteristics of actual smoking behavior, avoiding misjudging brief non-smoking actions such as bringing the hand close to the mouth as smoking behavior and reducing the false alarm rate.
[0070] Optionally, after determining that a person in the sub-image is smoking, the above-mentioned smoking behavior recognition method for the park may further include: locating the face region in the sub-image and extracting facial feature information from the face region; comparing the extracted facial feature information with a preset face database; if a match is found in the preset face database, obtaining the person's identity information; if a match fails to be found in the preset face database, assigning a unique identifier to the person; and associating the person's identity information or unique identifier with the smoking behavior recognition result and outputting the result.
[0071] The aforementioned facial feature extraction can be achieved through a deep learning embedding model. This model takes the cropped facial region image as input, preprocesses it, and then feeds it into a convolutional neural network backbone structure for feature encoding. One implementation of this scheme for facial feature extraction is as follows: The facial region is first located using a face detection algorithm (such as MTCNN or RetinaFace) to pinpoint key facial features, including the coordinates of the center of the eyes, the tip of the nose, and the corners of the mouth. The facial pose is then normalized to a standard frontal orientation through similarity or affine transformations, eliminating scale, rotation, and tilt differences, generating aligned facial image blocks. The aligned facial image is then input into a deep feature extraction network, which typically uses a residual network ResNet or Inception architecture as its backbone. Multi-level semantic features are extracted through multiple convolutional and pooling operations. After a global average pooling layer, a fully connected layer is added for dimensionality reduction, outputting a fixed-dimensional feature vector (e.g., 128-dimensional or 512-dimensional). During training, the extraction network is optimized using metric learning loss functions, including triplet loss or Angular Softmax loss (ArcFace Loss), to ensure that features of the same face are compactly spaced in the embedding space, while features of different faces are separated by distance. The extracted feature vectors are then L2 normalized to form a discriminative and compact facial feature representation. This representation is robust to changes in lighting, facial expressions, and mild occlusion, and can be directly used to perform cosine similarity or Euclidean distance comparisons with feature templates stored in a pre-defined face database to achieve identity recognition or verification.
[0072] When facial feature information fails to match a preset face database, a unique identifier allocation process is triggered. This process can generate a 128-bit identifier based on the UUID (Universally Unique Identifier) algorithm, combining the current timestamp, hardware device MAC address, and random number seed to ensure the global uniqueness and unpredictability of the identifier. Alternatively, a composite encoding method based on timestamp and hash function can be used. The current time value, accurate to milliseconds, is used to calculate a SHA-256 digest with the extracted facial feature vector, and the first 64 bits are truncated as the unique identifier for the person, ensuring the spatiotemporal uniqueness and anonymity of the identifier. The generated unique identifier and the extracted facial feature vector are bound and stored in a temporary feature database, establishing a mapping record between the identifier and the feature. At the same time, the timestamp of the first appearance and the video source location information are recorded, providing a basis for identity association for subsequent appearances of the person.
[0073] It should be noted that the personnel identity information stored in the preset face database is based on an authorized collection mechanism. This mechanism requires the data subject (park staff or registered visitors) to be clearly informed and sign a data collection authorization agreement before facial information can be entered. The stored identity information includes facial feature vectors and associated metadata. The facial feature vectors are obtained by performing deep feature extraction on the facial images of authorized personnel, and then forming a fixed-dimensional floating-point array after L2 normalization. The associated metadata includes structured fields such as personnel unique identifier, name, department, position, and permission level, supporting integration with the organizational structure of the management system. The data storage of the preset face database can be in an encrypted format. The facial feature vectors and metadata are encrypted using the AES-256 algorithm and stored in the database or hardware security module (HSM). The access interface implements a role-based access control (RBAC) policy, and only authorized management nodes or matching services can access the decryption key. In addition, the identity information collection process can follow the principle of minimum necessity, collecting only personnel information directly related to the security management of the park, and removing unnecessary sensitive fields during storage; at the same time, a data retention period mechanism is established, and when personnel leave the company or their authorization expires, their identity information is automatically marked as invalid and physically deleted after a preset period, ensuring that the face database storage complies with personal information protection regulations and park data security policies.
[0074] The above solution, by further locating the facial region and extracting feature information for identity comparison after identifying smoking behavior, realizes the correlation between behavior detection and identity recognition. This allows alarm events to be accurately linked to specific individuals, facilitating behavior tracing and responsibility confirmation for park management. On the other hand, the facial feature comparison mechanism can distinguish between registered personnel and temporary visitors in the preset facial database. For those who are successfully matched, identity information is directly obtained, while for those who are not matched, a unique identifier is automatically assigned, balancing the management efficiency of registered personnel with the complete recording of all detection targets.
[0075] The above smoking behavior recognition results can be output in structured data, and its content can include information on the following multiple dimensions: (1) Binary classification discrimination results, which indicate whether there is a smoking behavior in the human body in the form of a boolean value or a confidence score. This confidence score is obtained by thresholding the probability value output by the model, and its value range is between 0 and 1, reflecting the certainty level of the recognition algorithm. (2) Spatial positioning information, that is, the bounding box coordinates of the detected cigarette or the hand-cigarette interaction area, usually encoded in the form of the pixel coordinates of the upper left and lower right corners of the rectangular box, or the center point coordinates plus the width and height parameters, for visual annotation of the location where the smoking behavior occurs in the original image frame. (3) Timestamp information, as a key component of the recognition result, records the absolute time when the recognition event occurs or the frame number in the video stream, supporting subsequent event retrieval and statistical analysis according to the time dimension. (4) Human identity association information, which is the personnel identity information or the unique identifier assigned by the system, to bind the behavior event to a specific person.
[0076] Optionally, the above smoking behavior recognition method for the park can further include: generating a structured alarm event and reporting the structured alarm event to the management system; wherein, the structured alarm event includes one of the personnel identity information or the unique identifier, as well as the smoking behavior recognition result and the timestamp information; the reporting method of the structured alarm event includes local storage, network push or message notification; triggering the linkage response of the associated system based on the structured alarm event; wherein, the linkage response includes access control operation, voice broadcast or message push reminder.
[0077] The above structured alarm event is a standardized data object for encapsulating the complete semantic information of smoking behavior detection. This object is encoded in a hierarchical key-value pair structure or a binary protocol format, and includes mandatory identification fields, event fields and time fields. The identification field stores the human identity association information, which is dynamically filled with the personnel identity information or the unique identifier assigned by the system according to the face comparison result, and is represented in string or UUID binary form. The event field encodes the smoking behavior discrimination result, including a boolean existence flag bit and a floating-point confidence score. The timestamp information records the exact moment when the event occurs, using the Unix timestamp or the ISO 8601 standard time format. The structured alarm event supports multiple reporting mechanisms, including writing to a local log file for persistent storage, pushing to the management platform RESTful API via HTTP / HTTPS protocol POST request, or sending real-time message notifications to subscribed clients based on the MQTT / WebSocket protocol.
[0078] The following introduces the various linkage responses of the structured alarm event: (1) Access Control Operation: When a structured alarm event triggers the access control operation, the personnel location information (the physical area identifier obtained by converting the camera number and the detection frame coordinates) can be transmitted as a parameter to the access controller. The access controller executes corresponding actions according to the preset strategy: if the personnel are in the restricted area, the electric lock device is driven to remain locked and the abnormal access request is recorded; if the personnel are in the controllable area, the local audible and visual alarm at the access point is triggered, and a temporary lock command is sent to the card reader at the door point, prohibiting the identity identifier from swiping the card to open the door during the alarm period. The linkage implementation supports one-way and two-way control modes. In one-way mode, only personnel are prevented from entering the specific area. In two-way mode, the personnel's access channel is completely blocked until the management personnel manually cancel the alarm status through the management interface or the preset lock time expires and it automatically recovers.
[0079] (2) Voice Broadcast: By parsing the geographic area code of the camera, the system queries the broadcast area configuration table to obtain a list of IP speaker columns or horn speakers covered in that area, and generates a broadcast command containing warning text. The text content is converted into an audio stream by a speech synthesis engine and pushed to the target device via multicast or unicast. After the broadcast is triggered, the device repeatedly plays the warning voice at a preset volume. The number of broadcasts and the interval time are configurable until a stop command or alarm status clearing signal is received.
[0080] (3) Message Push Notification: After converting structured alarm events into human-readable notification content, they are pushed to different management terminals in parallel through multiple channels. For personnel on duty in fixed monitoring rooms, alarm information can be pushed to the client software interface in the form of a pop-up window. The pop-up window includes a real-time screenshot, personnel identity, incident location and timestamp, and remains on the screen until manual confirmation. For mobile management personnel, alarms can be forwarded to smartphone applications through push gateways, triggering vibration and ringtone alerts, while displaying an event summary in the notification bar. Clicking on the summary will redirect to the details page to view evidence screenshots and video playback links. For personnel who have not installed the dedicated application, alarm content can be sent through SMS gateways or email servers. The text includes key event elements and links to handling guidelines. Push implementation supports a hierarchical strategy, selecting push channel combinations according to personnel roles and alarm severity to ensure that critical alarms reach all relevant responsible personnel.
[0081] The above solution generates structured alarm events containing personnel identification, smoking behavior recognition results, and timestamps, and reports them through local storage, network push, or message notification. This achieves standardized encapsulation and multi-channel transmission of alarm information, ensuring that the management system can receive alarms in a timely and reliable manner. On the other hand, based on the linkage response mechanism triggered by the structured alarm events to access control, voice broadcast, or message push, the smoking behavior recognition results are transformed into actual park security control actions, forming a closed-loop management of detection, alarm, and response, and improving the automated handling capabilities of park security management.
[0082] It is understandable that after obtaining the smoking behavior recognition results, a visualization rendering operation can be performed on the original image frame to generate surveillance evidence footage with multi-layered annotations. For example: Figure 5 As shown, based on the bounding box coordinates output by the first-stage human target detection, a green rectangle is drawn at the corresponding position to mark the detected main body area of the person; for the person whose face recognition is successful, a red rectangle is superimposed on their face position and the person's identity information text is displayed; based on the cigarette target or hand-cigarette interaction area detected by the second-stage smoking behavior recognition model, a blue rectangle is drawn to accurately indicate the location where the smoking behavior occurs, and a category label and confidence score are marked next to the box. Figure 5 A magnified image of the detection box, as shown Figure 6 As shown, the green human detection bounding box covers the entire outline of the person, and the label outside the box reads "person: 0.61," indicating a 61% confidence level in detecting this human target. The red face recognition bounding box is located in the head area, covering the key facial features, and the label outside the box reads "name: NoName: -0.04," where the negative similarity score indicates that the person did not match successfully in the preset face database. The blue smoking behavior detection bounding box is located in the interaction area between the hands and face, and the label outside the box reads "sm: 0.28," indicating a 28% probability of smoking behavior occurring in this area.
[0083] like Figure 7 As shown, based on the same inventive concept, this application also provides a smoking behavior recognition system 200 for industrial parks, including a graphics processor 210 and an image acquisition module 220 communicatively connected to the graphics processor 210, wherein: Image acquisition module 220 is used to acquire video stream data from the park monitoring system and send the video stream data to graphics processor 210; The graphics processor 210 is used to acquire video stream data; decode the video stream data to obtain image frames; perform human target detection on the image frames to determine the location of each human body in the image frame; based on the location of the human body, crop out a sub-image containing the human body from the image frame; perform smoking behavior recognition on the sub-image to determine whether the human body in the sub-image is smoking; and output the smoking behavior recognition results of each human body in the image frame.
[0084] It is understood that the above-mentioned smoking behavior recognition system 200 for parks can realize any one of the functions of the smoking behavior recognition method for parks provided in the embodiments of this application. The methods and principles for realizing each function are described in the method embodiment section, and will not be repeated in the system embodiment section.
[0085] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of this application. (Refer to...) Figure 8 The electronic device 300 includes a processor 310, a memory 320, and a communication interface 330. These components are interconnected and communicate with each other via a communication bus 340 and / or other forms of connection mechanism (not shown).
[0086] The memory 320 includes one or more (only one is shown in the figure), which may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 310 and other possible components may access the memory 320 to read and / or write data therein.
[0087] Processor 310 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The processor 310 described above can be a general-purpose processor, including a central processing unit (CPU), a microcontroller unit (MCU), a network processor (NP), or other conventional processors; it can also be a special-purpose processor, including a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0088] Communication interface 330 includes one or more (only one is shown in the figure) that can be used to communicate directly or indirectly with other devices to exchange data. For example, communication interface 330 can be an Ethernet interface; it can be a mobile communication network interface, such as an interface for 3G, 4G, or 5G networks; or it can be other types of interfaces with data transmission and reception capabilities.
[0089] One or more computer program instructions may be stored in the memory 320. The processor 310 may read and run these computer program instructions to implement the smoking behavior recognition method for parks provided in the embodiments of this application and other desired functions.
[0090] Understandable. Figure 8 The structure shown is for illustrative purposes only; the electronic device 300 may also include components that are more advanced than those shown. Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown. Figure 8 The components shown can be implemented using hardware, software, or a combination thereof. For example, electronic device 300 can be a single server (or other device with computing power), a combination of multiple servers, a cluster of a large number of servers, etc., and can be either a physical device or a virtual device.
[0091] This application also provides a computer-readable storage medium storing computer program instructions. These instructions are read and executed by a processor to perform the smoking behavior recognition method for industrial parks provided in this application. For example, the computer-readable storage medium can be implemented as follows: Figure 8The memory 320 in the electronic device 300, or a separate storage product (such as a USB flash drive, portable hard drive, etc.).
[0092] This application also provides a computer program product, which includes computer program instructions. These computer program instructions are read and executed by a processor to perform the smoking behavior recognition method for industrial parks provided in this application. For example, these computer program instructions can be stored in... Figure 8 The memory 320 in the electronic device 300 is located inside the memory, or it is stored in a separate storage product (such as a USB flash drive, portable hard drive, etc.).
[0093] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0094] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0095] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0096] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method for recognizing smoking behavior in industrial parks, characterized in that, The method includes: Obtain video stream data from the park's surveillance cameras; The video stream data is decoded to obtain image frames; Human target detection is performed on the image frame to determine the location of each human body in the image frame; Based on the location of the human body, a sub-image containing the human body is cropped from the image frame; Smoking behavior recognition is performed on the sub-image to determine whether the human body in the sub-image is smoking; Output the smoking behavior recognition results of each person in the image frame.
2. The method for identifying smoking behavior in a park according to claim 1, characterized in that, The method further includes: The smoking recognition model is pre-trained on a publicly available smoking behavior dataset to learn a general feature representation of smoking behavior; wherein the publicly available smoking behavior dataset includes labeled images centered on the human body and in which cigarette butts are visible. The pre-trained smoking recognition model is fine-tuned based on the park scene annotation data so that the smoking recognition model can learn the image distribution characteristics under the park monitoring scene. The finely tuned smoking recognition model is used to identify smoking behavior in the sub-image.
3. The method for identifying smoking behavior in a park according to claim 1, characterized in that, The method further includes: The graphics processor is invoked to execute video stream data decoding, human target detection, sub-image cropping, and smoking behavior recognition steps; wherein the image frames, sub-images, and intermediate data generated in the human target detection and smoking behavior recognition steps are all stored in the graphics processor's video memory.
4. The method for identifying smoking behavior in a park according to claim 1, characterized in that, The process of cropping a sub-image containing the human body from the image frame based on the location of the human body includes: Get the bounding box of the area containing the human body; The bounding box is expanded to obtain an expanded bounding box; A sub-image is cropped from the image frame based on the expanded bounding box; wherein the relative proportion of human targets in the sub-image is higher than the relative proportion of human targets in the image frame.
5. The method for identifying smoking behavior in a park according to any one of claims 1 to 4, characterized in that, The method further includes: When there are multiple human bodies in the image frame, the cropping operation is performed on the area where each human body is located to obtain the sub-image corresponding to each human body. The smoking behavior recognition step is performed in parallel on the sub-images of each person to obtain the smoking behavior recognition result corresponding to each sub-image.
6. The method for identifying smoking behavior in a park according to any one of claims 1 to 4, characterized in that, The method further includes: Tracking the same human body across multiple consecutive image frames to determine the identity association of the same human body across multiple consecutive image frames; For the same human body, the smoking behavior recognition results of the human body in multiple consecutive image frames are counted as the number of frames in which smoking behavior exists; If the number of frames reaches a preset threshold, it is determined that the human body is smoking.
7. The method for identifying smoking behavior in a park according to any one of claims 1 to 4, characterized in that, After determining that a person in the sub-image is smoking, the method further includes: Locate the face region in the sub-image and extract facial feature information from the face region; The extracted facial feature information is compared with a preset face database. If a match is found in the preset face database, the person's identity information is obtained. If a match is not found in the preset face database, a unique identifier is assigned to the person. The personnel identity information or the unique identifier is associated with the smoking behavior identification result and output.
8. The method for identifying smoking behavior in a park according to claim 7, characterized in that, The method further includes: A structured alarm event is generated and reported to the management system; wherein, the structured alarm event includes either the personnel identity information or the unique identifier, as well as the smoking behavior identification result and timestamp information; the reporting method of the structured alarm event includes local storage, network push or message notification; The structured alarm event triggers a linkage response from the associated system; wherein the linkage response includes access control operation, voice broadcast or message push reminder.
9. A smoking behavior recognition system for industrial parks, characterized in that, It includes a graphics processor and an image acquisition module communicatively connected to the graphics processor, wherein: The image acquisition module is used to acquire video stream data from the park's surveillance system and send the video stream data to the graphics processor. The graphics processor is configured to: acquire the video stream data; decode the video stream data to obtain image frames; perform human target detection on the image frames to determine the location of each human body in the image frames; based on the location of the human body, crop out a sub-image containing the human body from the image frames; perform smoking behavior recognition on the sub-images to determine whether the human body in the sub-images is smoking; and output the smoking behavior recognition results of each human body in the image frames.
10. An electronic device, characterized in that, include: A processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; The memory stores program instructions that can be executed by the processor, and the processor can execute the method as described in any one of claims 1 to 8 by calling the program instructions.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1 to 8.
12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.
