Smoking hidden danger identification method, device and system
By constructing a deep learning model and fine-tuning it with incremental data, the accuracy and stability issues of identifying smoking hazards in industrial production sites were resolved, achieving uninterrupted automated identification at all times and improving identification accuracy and adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING INST OF RADIO METROLOGY & MEASUREMENT
- Filing Date
- 2025-12-08
- Publication Date
- 2026-05-01
AI Technical Summary
In existing technologies, the identification of smoking hazards in industrial production sites relies on manual inspections, which suffers from problems such as incomplete coverage, response delays, false positives and false negatives, and high labor costs. Furthermore, general target detection models are insufficient in terms of accuracy and generalization ability in identifying small targets.
By acquiring and preprocessing multi-source image data, a deep learning model is constructed to generate multi-level feature maps and perform sparse anchor point screening. Lightweight fine-tuning is then performed using incremental data to achieve real-time identification of smoking hazards in industrial sites.
It achieves uninterrupted automated identification of smoking hazards in industrial sites, improving identification accuracy and stability, making up for the omissions and delays of manual inspections, and adapting to changing environmental conditions.
Smart Images
Figure CN121963071A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and in particular to a method, apparatus and system for identifying smoking hazards. Background Technology
[0002] This section is intended to provide background or context for the embodiments of this application set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.
[0003] The production environments in industrial sectors such as chemical, energy, and manufacturing are characterized by flammable and explosive materials and densely distributed equipment. A lit cigarette, as an open flame source, can easily cause fires, explosions, and other safety accidents once it comes into contact with combustibles on site. This not only causes huge property losses but also seriously threatens the lives of workers and has irreversible negative impacts on the surrounding ecological environment and social stability. Currently, most industrial enterprises still rely on manual inspections as the core method for identifying smoking hazards. Specifically, this involves assigning safety management personnel to conduct on-site inspections of indoor operating rooms, outdoor storage tank areas, and open-air inspection passages at fixed or random times; or having designated personnel review surveillance footage in real time and review video recordings to visually identify whether anyone is smoking. Manual inspections are prone to missing smoking hazards due to incomplete coverage, staff fatigue, or lack of concentration. Furthermore, manual identification suffers from significant response delays; a considerable time lag often exists between the occurrence of smoking and the discovery of a hazard. If the cigarette butt comes into contact with flammable or explosive materials during this period, it can easily lead to a safety accident. Secondly, manual inspections are costly in terms of manpower, and the results are heavily influenced by subjective judgment, leading to misjudgments and missed detections, thus compromising the reliability of safety management. With the development of computer vision technology, some companies have attempted to use general object detection models for intelligent identification of smoking hazards. They collect a small amount of industrial site image data as training samples, directly input them into the general object detection model for training, and then input real-time image streams from on-site monitoring equipment into the model for inference, outputting the identification results of suspected smoking hazards. While this type of intelligent method reduces reliance on manual labor to some extent, it still suffers from insufficient accuracy in small target recognition and poor model generalization ability. Summary of the Invention
[0004] One objective of this application is to provide a method for identifying smoking hazard, another objective is to provide a device for identifying smoking hazard, and yet another objective is to provide a system for identifying smoking hazard.
[0005] To achieve the above objectives, this application discloses a method for identifying smoking hazards, the method comprising: Multi-source image data is acquired and preprocessed to obtain a training dataset. The training dataset is then input into a deep learning model containing a path aggregation network. By generating multi-level feature maps and performing sparse anchor point filtering, the model training is completed, and an initial smoking hazard identification model is obtained. Real-time smoking behavior images are collected periodically as incremental data. The incremental data is fused with the training dataset to obtain an iterative dataset. Based on the iterative dataset, the small target detection head of the initial smoking hazard identification model is lightly fine-tuned to obtain an updated smoking hazard identification model. The real-time image stream from the industrial site is input into the updated smoking hazard identification model, inference analysis is performed, and the smoking hazard identification result is output.
[0006] Optionally, acquiring multi-source image data includes: Collect image data from different scenes, which at least include indoor scenes, outdoor scenes, daytime scenes, and nighttime scenes; Collect image data acquired by different shooting devices, which at least include fixed-view shooting devices, rotatable-view shooting devices, and mobile inspection shooting devices; Image data under different lighting conditions are collected, and the lighting conditions at least cover natural light, low light, and supplemental light.
[0007] Optionally, generating multi-level feature maps includes: The training dataset is input into the path aggregation network; The path aggregation network outputs a first feature map and a second feature map. The first feature map is a low-semantic high-resolution feature map used for small target reasoning, and the second feature map is a high-semantic low-resolution feature map used for ordinary target reasoning.
[0008] Optionally, the sparse anchor filtering process includes: Based on the second feature map, a sparse probability map is generated, wherein the value of each pixel in the sparse probability map represents the probability that a small target exists at the corresponding location; Filter the anchor points corresponding to pixels whose values in the sparse probability map satisfy the preset small target existence probability threshold, and use them as positive sample anchor points for small targets. For the dense branch anchor points corresponding to the first feature map, determine whether they simultaneously meet the preset conditions, filter the anchor points that meet the preset conditions as ordinary target positive sample anchor points, and summarize the small target positive sample anchor points and the ordinary target positive sample anchor points to obtain the filtered positive sample anchor point set.
[0009] Optionally, the preset conditions include: The anchor point center is located within a preset pixel range of the actual bounding box; The alignment index between the anchor point and the actual bounding box ranks among the top N preset values in all anchor boxes; The intersection-union ratio between the anchor point and the actual bounding box is the maximum.
[0010] Optionally, completing the model training to obtain the initial smoking hazard identification model includes: The training dataset is divided into a training subset and a validation subset according to a preset ratio; Using the pre-trained weights obtained from training on a general object detection dataset as initial weights, the training subset and the set of positive sample anchor points are input into the deep learning model to train the backbone network and prediction head of the model. The validation subset is input into the trained model to evaluate the model performance and adjust the hyperparameters until the model performance reaches a preset performance threshold. The adjusted model is then used as the initial smoking hazard identification model.
[0011] Optionally, the periodic collection of real-time smoking behavior images from the industrial site as incremental data includes: Set a preset time interval for data collection, and perform data collection operations according to the preset time interval; Smoking behavior images are captured in real time using monitoring equipment at the industrial site. The images include at least those taken under supplementary lighting during night shifts. All captured images are uploaded to the server as incremental data.
[0012] Optionally, the lightweight fine-tuning of the small target detection head of the initial smoking hazard identification model includes: Fix the backbone network parameters of the initial smoking hazard identification model; The iterative dataset is input into the initial smoking hazard identification model, and only the small target detection head parameters of the model are adjusted. The small target detection head is a convolutional module. Set a fine-tuning learning rate, with the initial learning rate set to a preset initial learning rate. After each preset training round, reduce the learning rate by a preset ratio and continue training until the model loss value is stable and less than a preset loss threshold to obtain the updated smoking hazard identification model.
[0013] Another aspect of this application discloses a smoking hazard identification device, the device comprising: Preprocessing module: Acquires and preprocesses multi-source image data to obtain a training dataset. The training dataset is then input into a deep learning model containing a path aggregation network. By generating multi-level feature maps and performing sparse anchor point filtering, the model training is completed, and an initial smoking hazard identification model is obtained. Lightweight fine-tuning module: Periodically collect real-time smoking behavior images as incremental data, fuse the incremental data with the training dataset to obtain an iterative dataset, and perform lightweight fine-tuning on the small target detection head of the initial smoking hazard identification model based on the iterative dataset to obtain an updated smoking hazard identification model; Inference Analysis Module: Input the real-time image stream from the industrial site into the updated smoking hazard identification model, perform inference analysis, and output the smoking hazard identification results.
[0014] Another aspect of this application discloses a smoking hazard identification system, the system comprising the smoking hazard identification device described above.
[0015] The beneficial effects of this application are as follows: This application utilizes multi-source image data to drive the construction of a deep learning model and combines it with real-time image stream inference analysis. This enables continuous and automated identification of smoking behavior in industrial settings, with immediate output of identification results. This allows safety managers to quickly locate and address potential hazards. By acquiring and preprocessing multi-source image data covering multiple scenes, multiple shooting devices, and multiple lighting conditions, the feature dimensions of the training samples are enriched, allowing the trained model to adapt to changing environments. Multi-level feature maps are generated based on path aggregation networks, taking into account both high-resolution detail features of small targets and high semantic features of ordinary targets. The effectiveness of anchor points is optimized by combining a sparse anchor point screening strategy, solving the problems of high false negative rates for small cigarette butts and computational redundancy caused by dense anchor points in traditional target detection models. This significantly improves the accuracy and stability of smoking hazard identification. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This document shows a flowchart illustrating a specific embodiment of the smoking hazard identification method according to this application. Figure 2 The flowcharts of specific embodiments S110-S130 of the smoking hazard identification method of this application are shown. Figure 3 The flowcharts of specific embodiments S140-S150 of the smoking hazard identification method of this application are shown. Figure 4 The flowcharts of specific embodiments S160-S180 of the smoking hazard identification method of this application are shown. Figure 5 The flowcharts of specific embodiments S181-S183 of the smoking hazard identification method of this application are shown. Figure 6 The flowcharts of specific embodiments S210-S230 of the smoking hazard identification method of this application are shown. Figure 7 The flowcharts of specific embodiments S240-S250 of the smoking hazard identification method of this application are shown. Figure 8 The flowcharts of specific embodiments S260-S280 of the smoking hazard identification method of this application are shown. Figure 9 This diagram illustrates the structure of a specific module of the smoking hazard identification device according to an embodiment of this application. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.
[0018] It should be understood that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "include" and "have," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or programmable logic device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or programmable logic devices.
[0019] References to embodiments in this application mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.
[0020] In order to solve at least one of the problems existing in the prior art, according to one aspect of this application, such as Figure 1 As shown, this embodiment discloses a method for identifying smoking hazards, such as... Figure 1 As shown, the method includes: S100: Acquire multi-source image data and preprocess it to obtain a training dataset. Input the training dataset into a deep learning model containing a path aggregation network. By generating multi-level feature maps and performing sparse anchor point filtering, complete the model training and obtain the initial smoking hazard identification model.
[0021] Specifically, real-time identification of smoking hazards in industrial sites is achieved through the construction of a deep learning model driven by multi-source data and combined with lightweight updates of incremental data. First, multi-source image data is acquired and preprocessed to obtain a training dataset. This training dataset is then input into a deep learning model containing a path aggregation network. Multi-level feature maps are generated, and sparse anchor point filtering is performed to complete model training, resulting in an initial smoking hazard identification model. Real-time smoking behavior images are periodically collected as incremental data, which is then fused with the training dataset to obtain an iterative dataset. Based on this iterative dataset, the small target detection head of the initial smoking hazard identification model is lightweightly fine-tuned to obtain an updated smoking hazard identification model. Real-time image streams from the industrial site are then input into the updated smoking hazard identification model for inference analysis, outputting the smoking hazard identification results.
[0022] It should be noted that multi-source image data refers to a collection of image data acquired from different dimensions and scenarios. The acquisition process covers various areas in industrial settings where smoking may occur. Acquisition methods include, but are not limited to, continuous shooting by surveillance cameras and mobile inspection snapshots. The purpose of acquisition is to provide a rich sample base for model training, avoiding insufficient model generalization ability due to a single sample. Preprocessing involves performing a series of image enhancement operations on the acquired multi-source image data. By changing the form of the images, hidden feature information is represented. The preprocessed image set forms the training dataset, which is the basis for the model to learn the characteristics of smoking hazards.
[0023] Path aggregation networks are neural network structures used for feature fusion. They arose from the need to optimize the utilization of feature information at different levels in object detection. They aggregate shallow, high-resolution features with deep, semantically rich features, thus balancing feature extraction for both small and large targets. Deep learning models are object detection models built upon path aggregation networks, combined with backbone networks, prediction heads, and other modules. The training process involves the model learning to distinguish between smoking hazard features and non-hazard features related to cigarette butts and smoking actions in the training dataset. Multi-level feature maps represent different dimensions of features output by the path aggregation network after processing the training dataset. Sparse anchor point selection is an optimized strategy for selecting target anchor points in the feature maps, addressing the high computational cost caused by dense anchor points. Its working principle involves using probability filtering and conditional judgment to retain only high-value anchor points for model training. The final trained model is the initial smoking hazard identification model, which possesses basic smoking hazard identification capabilities.
[0024] S200: Periodically collect real-time smoking behavior images as incremental data, fuse the incremental data with the training dataset to obtain an iterative dataset, and perform lightweight fine-tuning on the small target detection head of the initial smoking hazard identification model based on the iterative dataset to obtain an updated smoking hazard identification model.
[0025] Specifically, incremental data consists of real-time images of smoking behavior continuously collected from industrial sites after model deployment. The collection frequency matches the safety management needs of industrial sites. The iterative dataset formed by merging incremental data with the original training dataset can supplement new scene features that the model has not learned. The small target detection head is a module in the deep learning model specifically used to identify small targets such as cigarette butts. The lightweight fine-tuning is to solve the problem of time-consuming and computationally expensive full retraining. By fixing the basic parameters of the model and only updating the parameters of the small target detection head, it can quickly adapt to new scenarios. The updated smoking hazard identification model obtained after fine-tuning can balance recognition accuracy and update efficiency.
[0026] S300: Input the real-time image stream from the industrial site into the updated smoking hazard identification model, perform inference analysis, and output the smoking hazard identification result.
[0027] Specifically, a real-time image stream is a continuous sequence of images transmitted in real time by industrial site monitoring equipment. After inputting the updated model, the model performs inference analysis, which is the process by which the model applies the learned feature rules to the new images. The final output of the smoking hazard identification result includes information such as the location and category of the hazard, which can directly support safety management decisions in industrial sites. It can realize the automatic real-time identification of smoking hazards, thereby making up for the omissions and delays of manual inspections.
[0028] In alternative implementations, such as Figure 2 As shown, acquiring multi-source image data includes: S110: Collect image data of different scenes, which at least cover indoor scenes, outdoor scenes, daytime scenes and nighttime scenes.
[0029] Specifically, the process of acquiring multi-source image data aims to enable the model to cover various complex scenarios in industrial settings, thereby improving the model's generalization ability. This includes collecting image data from different scenarios, images acquired by different shooting devices, and images under different lighting conditions. Different scenarios include indoor scenarios, outdoor scenarios, daytime scenarios, and nighttime scenarios. Indoor scene image data is collected from enclosed or semi-enclosed areas such as factory buildings, control rooms, and rest rooms in industrial settings. Outdoor scene image data is collected from open areas such as factory inspection passages, tank areas, and open storage yards. Daytime and nighttime scene image data are collected under natural light and no natural light conditions, respectively. This type of data allows the model to learn the characteristics of smoking hazards in different spatial environments, avoiding the model being adapted to only a single scenario.
[0030] S120: Collect image data acquired by different shooting devices, wherein the shooting devices include at least fixed-view shooting devices, rotatable-view shooting devices, and mobile inspection shooting devices.
[0031] Specifically, the image data acquired by different shooting devices includes images collected by fixed-view shooting devices, rotatable-view shooting devices, and mobile inspection shooting devices. Fixed-view shooting devices are usually bullet cameras permanently installed in the factory area, which capture images with a stable viewing angle and can cover a fixed area. Rotatable-view shooting devices are usually PTZ cameras, which can adjust the shooting angle and focal length through remote control, and the captured images can cover a wider range. Mobile inspection shooting devices are terminal devices held by inspection personnel, and the captured images can supplement the blind spots covered by fixed cameras. This type of data is used to allow the model to adapt to image features of different shooting angles and resolutions, and improve the model's compatibility with data collected by different devices.
[0032] S130: Collect image data under different lighting conditions, wherein the lighting conditions at least cover natural light illumination, low light illumination, and supplementary light illumination.
[0033] Specifically, the image data under different lighting conditions includes images under natural light, low light, and supplemental lighting conditions. Images under natural light are collected in unobstructed daytime lighting environments, images under low light are collected in environments with insufficient lighting such as cloudy days or nighttime without supplemental lighting, and images under supplemental lighting are collected in environments with auxiliary lighting such as infrared supplemental lighting or fluorescent lighting at night. The data collection process needs to be combined with real-time data from the light sensor to label the lighting type. This allows the model to learn the characteristic changes of smoking hazards under different lighting conditions, such as the brightness characteristics of cigarette butts under low light and the reflective characteristics under supplemental lighting. This avoids recognition failure due to differences in lighting and ultimately improves the model's recognition stability in various lighting scenarios.
[0034] In alternative implementations, such as Figure 3 As shown, the generation of multi-level feature maps includes: S140: Input the training dataset into the path aggregation network.
[0035] Specifically, generating multi-level feature maps involves splitting and optimizing the training dataset into feature levels using a path aggregation network, thus balancing feature extraction performance for both small and large targets. It's important to note that the first step in generating multi-level feature maps is inputting the training dataset into the path aggregation network. This network, acting as a feature fusion network, addresses the problem in object detection where shallow features have high resolution but insufficient semantic information, while deep features possess rich semantic information but low resolution. Its principle is to achieve the fusion of features from different levels through top-down feature propagation and bottom-up feature enhancement.
[0036] S150: The path aggregation network outputs a first feature map and a second feature map. The first feature map is a low-semantic high-resolution feature map for small target reasoning, and the second feature map is a high-semantic low-resolution feature map for ordinary target reasoning.
[0037] Specifically, after processing the training dataset, the path aggregation network outputs a first feature map and a second feature map. The first feature map is a low-semantic, high-resolution feature map. Low semantics means that this feature map contains low-level visual features of the image, such as edges and textures. High resolution means that this feature map preserves the detailed information of the image. Its generation process is that the path aggregation network performs shallow feature extraction and resolution preservation on the training dataset. The first feature map is used for small object reasoning because the features of small objects, such as cigarette butts, are mainly reflected in low-level visual details, and high resolution can better capture these detailed features. The second feature map is a high-semantic, low-resolution feature map. High semantics means that this feature map contains high-level semantic features of the image, such as object category and contextual relationships. Low resolution is the result after compressing redundant information during feature extraction. Its generation process is that the path aggregation network performs deep feature extraction and semantic information integration on the training dataset. The principle of this feature map is to be used for ordinary object reasoning. Ordinary objects, such as hands and jaws, are relatively large in size, and their features are reflected at the semantic level. The low-resolution, high-semantic feature map can capture the features of such objects while reducing computational costs, ultimately enabling the model to have the feature extraction capabilities for both small and ordinary objects, improving the overall recognition accuracy.
[0038] In alternative implementations, such as Figure 4 As shown, the sparse anchor point filtering process includes: S160: Based on the second feature map, a sparse probability map is generated, wherein the value of each pixel in the sparse probability map represents the probability that a small target exists at the corresponding position.
[0039] Specifically, sparse anchor point screening involves using probability filtering and conditional judgment to select high-value positive sample anchor points from the anchor points in the feature map, avoiding the waste of computational resources and increased false detection rate caused by dense anchor points. The first step in sparse anchor point screening is to generate a sparse probability map based on the second feature map. The sparse probability map is a matrix obtained by probabilistically calculating the semantic information of the second feature map. The value of each pixel in the matrix represents the probability that a small target exists at the corresponding location. The calculation process of this value combines dimensions such as the semantic feature matching degree and positional feature correlation of the small target, and its purpose is to provide a quantitative basis for the screening of small target anchor points.
[0040] S170: Select the anchor points corresponding to the pixels whose values in the sparse probability map meet the preset small target existence probability threshold, and use them as positive sample anchor points for small targets.
[0041] Specifically, based on the values of the sparse probability map, anchor points corresponding to pixels that meet the preset small target existence probability threshold are selected as positive small target sample anchor points. The preset small target existence probability threshold is set based on the requirement of balancing recall and accuracy in small target recognition. The screening process essentially retains anchor points with probability values higher than the threshold. The regions corresponding to these anchor points are likely to contain small targets such as cigarette butts, reducing invalid anchor points participating in training and improving the efficiency of small target recognition.
[0042] S180: For the dense branch anchor points corresponding to the first feature map, determine whether they simultaneously meet the preset conditions, filter the anchor points that meet the preset conditions as ordinary target positive sample anchor points, summarize the small target positive sample anchor points and the ordinary target positive sample anchor points to obtain the filtered positive sample anchor point set.
[0043] Specifically, for dense branch anchors corresponding to the first feature map, it is necessary to determine whether they simultaneously meet preset conditions. Anchors that meet the conditions are selected as ordinary target positive sample anchors. Dense branch anchors are a set of anchors that cover the first feature map, and their number is related to the feature map resolution. The preset conditions are validity judgment rules for ordinary target anchors such as hands and jaws. The selection process verifies whether each anchor meets all preset conditions and only retains those that do. It should be noted that the final positive sample anchors for small targets and ordinary targets are summarized to obtain the selected set of positive sample anchors. This set is used as the basis for calculating loss and updating parameters during model training. This achieves anchor sparsity, allowing the model to focus on the target features corresponding to high-value anchors, ultimately reducing the computational cost of model training while improving the accuracy of target recognition.
[0044] In alternative implementations, such as Figure 5 As shown, the preset conditions include: S181: The anchor point center is located within the preset pixel range of the actual bounding box.
[0045] Specifically, the preset condition for sparse anchor point screening is the basis for determining whether dense branch anchor points are valid positive sample anchor points for ordinary targets. The screening of anchor points is based on the matching degree between the anchor points and the real target bounding boxes to ensure that the selected anchor points can represent the features of ordinary targets.
[0046] The anchor point center is located within a preset pixel range of the ground truth bounding box. The anchor point center is the coordinate center point of the anchor point in the feature map, calculated by the model based on the anchor point's size and position. The ground truth bounding box is a bounding box obtained by labeling common targets such as hands and jaws in the training dataset. Its coordinate range corresponds to the actual position of the target in the image. This condition ensures that the anchor point's position coincides with the position of the ground truth target, thereby avoiding feature learning bias caused by the anchor point deviating from the target.
[0047] S182: The alignment index of the anchor point with the actual bounding box ranks in the top N of all anchor boxes by a preset value.
[0048] Specifically, the alignment metric between anchor points and the ground truth bounding boxes is ranked among the top positions of all anchor points. This alignment metric is a quantitative measure of the degree of overlap and directional matching between anchor points and the ground truth bounding boxes, calculated by combining dimensions such as the overlap area and aspect ratio matching. It selects the anchor points with the best alignment with the ground truth from all candidate anchor points, thus avoiding feature interference caused by low-matching anchor points participating in training.
[0049] S183: The intersection-union ratio between the anchor point and the actual bounding box is the maximum.
[0050] Specifically, the intersection-union ratio (IUGR) is the ratio of the intersection area of the bounding box corresponding to the anchor point to the union area of the ground truth bounding box. It is calculated by obtaining the intersection and union regions of the two bounding boxes through coordinate operations, and then calculating the area ratio. The principle behind this condition is to ensure that each ground truth bounding box is matched with only the optimal anchor point, avoiding training chaos caused by one target corresponding to multiple anchor points. By using the above conditions, accurate positive sample anchor points for ordinary targets are selected, thereby improving the model's recognition accuracy for ordinary targets.
[0051] In alternative implementations, such as Figure 6 As shown, the process of completing model training to obtain the initial smoking hazard identification model includes: S210: Divide the training dataset into a training subset and a validation subset according to a preset ratio.
[0052] Specifically, the process of completing model training and obtaining an initial smoking hazard identification model involves combining the training dataset with a selected set of positive sample anchors. Through phased training and evaluation, the model develops a stable ability to identify smoking hazard. It's important to note that the training dataset is first divided into a training subset and a validation subset according to a predetermined ratio. This predetermined ratio is based on the need to balance the sufficiency of training samples with the effectiveness of evaluation. The division process involves randomly selecting a portion of the training dataset as the validation subset, and the remaining samples as the training subset. The principle behind this division is to use the training subset to learn the model parameters and the validation subset to evaluate the model's generalization ability, thereby avoiding overfitting.
[0053] S220: Using the pre-trained weights obtained from training on the general object detection dataset as initial weights, the training subset and the set of positive sample anchor points are input into the deep learning model to train the backbone network and prediction head of the model.
[0054] Specifically, the pre-trained weights obtained from training on a general object detection dataset are used as initial weights. The training subset and the set of positive sample anchor points are input into the deep learning model to train the backbone network and prediction head of the model. The general object detection dataset is a publicly available labeled dataset in the field that contains multiple types of objects. The pre-trained weights are the parameter set of the general feature extraction capabilities learned by the model on this dataset. They are obtained by directly calling the publicly available model parameter file. The backbone network is the module in the model responsible for extracting basic image features, and the prediction head is the module responsible for outputting the target category and location based on the features. The principle of the training process is to allow the model to learn the specific features of smoking hazards based on the pre-trained weights, reducing the time spent training from scratch.
[0055] S230: Input the validation subset into the trained model, evaluate the model performance and adjust the hyperparameters until the model performance reaches the preset performance threshold, and use the adjusted model as the initial smoking hazard identification model.
[0056] Specifically, the validation subset is input into the trained model to evaluate its performance and adjust hyperparameters until the model performance reaches a preset performance threshold. The evaluation dimensions of model performance include accuracy, recall, and false positive rate. The evaluation process involves inputting the validation subset into the model and comparing the model's output with the labeled results. Hyperparameters are parameters manually set during model training, such as learning rate and number of iterations. Their adjustment is based on the evaluation results of the validation subset; for example, if the recognition accuracy is low, the learning rate is adjusted to improve feature learning efficiency. The preset performance threshold is set based on the actual needs of identifying smoking hazards in industrial settings. A model that reaches this threshold possesses practical recognition capabilities. Finally, the adjusted model is used as the initial smoking hazard identification model, achieving the effect of having basic recognition accuracy while possessing good generalization ability.
[0057] In alternative implementations, such as Figure 7 As shown, the periodic collection of real-time smoking behavior images from industrial sites as incremental data includes: S240: Set a preset time interval for data collection, and perform data collection operations according to the preset time interval.
[0058] Specifically, regularly collecting real-time images of smoking behavior in industrial settings as incremental data is a prerequisite for continuous model optimization. This allows the model to supplement new scenarios and feature samples not covered by the existing model, enabling it to adapt to dynamic changes in the industrial environment. It's important to note that a preset data collection interval is first set, and data collection is performed at this interval. This preset interval is determined by the frequency of smoking behavior in the industrial setting and the rate of change in the scene; for example, the interval can be shortened during shift adjustments and extended during stable production periods. The data collection operation is executed by the industrial site monitoring system, which triggers image acquisition commands at the preset intervals.
[0059] S250: Real-time capture of smoking behavior images using monitoring equipment at the industrial site, including at least images taken under supplementary lighting during night shifts, and uploading all captured images to the server as incremental data.
[0060] Specifically, images of smoking behavior are captured in real time by monitoring equipment in the industrial site. These images include at least those taken under supplemental lighting during night shifts. The monitoring equipment includes fixed cameras and mobile inspection devices already deployed in the factory area. Images taken under supplemental lighting during night shifts are collected in monitored areas where infrared or daylight illumination is turned on at night. During the acquisition process, information such as the shooting time, area, and lighting type is simultaneously labeled. Its purpose is to supplement the model's samples under nighttime supplemental lighting scenarios, solving the problem that the original model was only suitable for daytime scenarios. All captured images are uploaded to a server as incremental data. The server is a hardware device used to store image data. The image upload process is realized through the industrial network. After uploading, the images are formatted and labeled with information such as the acquisition time and equipment number. The principle of incremental data is to provide new feature samples for subsequent lightweight fine-tuning of the model, so that the model can adapt to the scene characteristics newly added in the industrial site, such as nighttime supplemental lighting and newly put into production areas, avoiding the degradation of model performance as the scene changes.
[0061] In alternative implementations, such as Figure 8 As shown, the lightweight fine-tuning of the small target detection head of the initial smoking hazard identification model includes: S260: Fix the backbone network parameters of the initial smoking hazard identification model.
[0062] Specifically, the lightweight fine-tuning of the small target detection head of the initial smoking hazard identification model allows the model to adapt to incremental data features without retraining the entire model. The principle is to focus on the small target recognition module of the model, achieving rapid parameter updates. First, the backbone network parameters of the initial smoking hazard identification model are fixed. These backbone network parameters are the set of parameters in the model responsible for extracting general image features. These parameters have already learned stable feature extraction capabilities during initial training, and fixing their parameters avoids the computational and time costs associated with full retraining. The small target detection head parameters are parameters in the model specifically used for recognizing small targets such as cigarette butts. Maintaining their adjustability allows the model to be optimized only for small target recognition capabilities, adapting to new small target features in the incremental data.
[0063] S270: Input the iterative dataset into the initial smoking hazard identification model, and adjust only the small target detection head parameters of the model, wherein the small target detection head is a convolutional module.
[0064] Specifically, the iterative dataset is input into the initial smoking hazard identification model, and only the parameters of the small target detection head of the model are adjusted. The small target detection head is a convolutional module, which is a network structure composed of multiple convolutional layers. Its working principle is to extract local features of small targets through convolution operations. The process of adjusting the parameters of this module is to calculate the loss value of the iterative dataset and then update the parameters through backpropagation. Updating only the parameters of this module can significantly reduce the amount of computation.
[0065] S280: Set a fine-tuning learning rate. The initial learning rate is set to a preset initial learning rate. The learning rate is reduced by a preset ratio after each preset training round. The training continues until the model loss value is stable and less than the preset loss threshold, thus obtaining the updated smoking hazard identification model.
[0066] Specifically, the initial learning rate is set to a preset initial learning rate. Each preset training epoch reduces the learning rate by a preset percentage. Fine-tuning the learning rate controls the magnitude of parameter updates. The preset initial learning rate is set based on the degree of feature difference between the incremental data and the original dataset; a larger feature difference results in a slightly higher initial learning rate, and vice versa. The principle behind reducing the learning rate each preset training epoch is to use a higher learning rate in the early stages of training to quickly adapt to new features, and a lower learning rate in the later stages to finely adjust parameters and avoid parameter oscillations. Training continues until the model loss value stabilizes and is less than a preset loss threshold. The loss value is a quantitative indicator that measures the difference between the model's output and the actual annotations. Its calculation combines classification loss and localization loss. The preset loss threshold is set based on the accuracy requirements for small target recognition. Reaching this threshold indicates that the model has adapted to the incremental data features. The resulting updated smoking hazard recognition model combines recognition accuracy and update efficiency.
[0067] In a specific example, a chemical production plant encompasses various areas, including indoor operating rooms, outdoor tank areas, and open-air inspection channels. Production relies on natural light during the day, while infrared supplemental lighting is activated at night to ensure smooth operations. To address the issue of missed detections during manual inspections, a smoking hazard identification system was deployed. The first step involved acquiring and preprocessing multi-source image data to generate a training dataset. The plant first collected data for different scenarios, covering daytime natural light conditions in indoor operating rooms and outdoor tank areas, low-light conditions without supplemental lighting at night, and nighttime infrared supplemental lighting conditions. Secondly, fixed-view bullet cameras captured images of fixed areas in the tank area, rotating PTZ cameras captured images of a wide area in the inspection channels, and mobile inspection equipment captured images of camera blind spots. Finally, images were simultaneously acquired under different lighting conditions, including direct natural light, natural light obstruction (cloudy days), low light, and infrared supplemental lighting. Preprocessing is performed on the above multi-source image data. First, multiple images are randomly cropped and scaled. After cropping a specified area, they are rearranged and stitched into a single image. The stitched image is then randomly subjected to affine transformations involving rotation, translation, scaling, and shearing. Two images, including the image after the affine transformation, are selected, their transparency is adjusted, and they are then merged to complete image blending enhancement. Finally, the fused image has its chroma, saturation, and transparency randomly adjusted, and is flipped vertically or horizontally. All processed images are then combined to form a training dataset. The second step involves generating multi-level feature maps and filtering sparse anchor points to complete the initial model training. The training dataset is input into the deep learning model with a built-in path aggregation network. After receiving the training dataset, the path aggregation network outputs a first feature map and a second feature map. Based on the second feature map, a sparse probability map is generated. Pixels whose values satisfy a preset small target existence probability threshold are selected as anchor points for positive samples of small targets. For the dense branch anchor points corresponding to the first feature map, it is verified whether they simultaneously satisfy the following conditions: the anchor point center is located within a preset pixel range of the true bounding box, the alignment index ranks in a preset order, and the intersection-union ratio is the largest. The process involves several steps: 1) Selecting positive anchor points for common targets and summarizing both types of anchor points to obtain a set of positive anchor points. 2) Dividing the training dataset into training and validation subsets according to a preset ratio. Using pre-trained weights obtained from the general object detection dataset as initial weights, the training subset and the set of positive anchor points are input into the model to train a backbone network for extracting basic features and a prediction head for outputting target category / location. The validation subset is then input into the trained model to evaluate performance metrics such as accuracy and recall, and to adjust hyperparameters such as learning rate and number of iterations until the model performance reaches a preset threshold, thus obtaining the initial smoking hazard identification model. 3) Incremental data collection and fusion to generate an iterative dataset.The factory has added a continuous night shift production mode. Under infrared supplementary lighting at night, the characteristics of workers smoking—such as hands obscuring their faces and the reflective features of cigarette butts—are significantly different from those during the day shift. The system initiates an incremental data collection process: Data collection commands are triggered at preset time intervals, and images of smoking behavior under infrared supplementary lighting during the night shift are captured in real-time through factory monitoring equipment. The images are simultaneously labeled with the shooting area, lighting type, and other information before being uploaded to the server. For the newly added images stored on the server, the same preprocessing operation as in the first step is performed. The preprocessed incremental data is then merged with the original training dataset, and duplicate images are removed to generate an iterative dataset. The fourth step involves lightweight model fine-tuning to generate an updated recognition model. The system calls the lightweight fine-tuning module to optimize the initial model, fixing the backbone network parameters of the initial model and only granting access to adjust the parameters of the small target detection head. The iterative dataset is input into the initial model, and only the parameters of the small target detection head are adjusted. An initial learning rate is set for fine-tuning, and the learning rate is reduced by a preset percentage after each preset training epoch. Training continues until the model loss value stabilizes and falls below a preset threshold, resulting in an updated smoking hazard recognition model. The fifth step involves real-time image stream inference analysis to output the hazard recognition results. The plant's monitoring equipment continuously transmits real-time image streams of the outdoor tank area under infrared illumination at night to the system's inference and analysis module. This module breaks down the image stream into individual frames, inputting them into an updated model. The model then outputs multi-level feature maps through a path aggregation network, repeating sparse anchor point filtering logic to identify the presence of smoking hazards in the images. The model outputs the location and category of the hazard, transmitting this information to the central control room display terminal. Safety management personnel can visually see the marked hazard areas and immediately proceed to the site for handling. The entire process achieves real-time identification of smoking hazards without human intervention.
[0068] Another aspect of this application discloses a smoking hazard identification device, such as... Figure 9 As shown, the device includes: Preprocessing module 11: Acquires multi-source image data and performs preprocessing to obtain a training dataset. The training dataset is then input into a deep learning model containing a path aggregation network. By generating multi-level feature maps and performing sparse anchor point filtering, the model training is completed, and an initial smoking hazard identification model is obtained. Lightweight fine-tuning module 12: Periodically collects real-time smoking behavior images as incremental data, merges the incremental data with the training dataset to obtain an iterative dataset, and performs lightweight fine-tuning on the small target detection head of the initial smoking hazard identification model based on the iterative dataset to obtain an updated smoking hazard identification model. Reasoning and analysis module 13: Input the real-time image stream from the industrial site into the updated smoking hazard identification model, perform reasoning and analysis, and output the smoking hazard identification results.
[0069] Since the principle by which this device solves the problem is similar to the methods described above, the implementation of this device can be found in the implementation of the methods, and will not be repeated here.
[0070] In another aspect, this application discloses a smoking hazard identification system, which includes the smoking hazard identification device described above.
[0071] Since the principle by which this system solves the problem is similar to the methods and apparatus described above, the implementation of this apparatus can be found in the implementation of the methods and apparatus, and will not be repeated here.
[0072] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0073] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.
Claims
1. A method for identifying smoking hazard, characterized in that, The method includes: Multi-source image data is acquired and preprocessed to obtain a training dataset. The training dataset is then input into a deep learning model containing a path aggregation network. By generating multi-level feature maps and performing sparse anchor point filtering, the model training is completed, and an initial smoking hazard identification model is obtained. Real-time smoking behavior images are collected periodically as incremental data. The incremental data is fused with the training dataset to obtain an iterative dataset. Based on the iterative dataset, the small target detection head of the initial smoking hazard identification model is lightly fine-tuned to obtain an updated smoking hazard identification model. The real-time image stream from the industrial site is input into the updated smoking hazard identification model, inference analysis is performed, and the smoking hazard identification result is output.
2. The smoking hazard identification method according to claim 1, characterized in that, The acquisition of multi-source image data includes: Collect image data from different scenes, which at least include indoor scenes, outdoor scenes, daytime scenes, and nighttime scenes; Collect image data acquired by different shooting devices, which at least include fixed-view shooting devices, rotating-view shooting devices, and mobile inspection shooting devices; Image data under different lighting conditions are collected, and the lighting conditions at least cover natural light, low light, and supplemental light.
3. The smoking hazard identification method according to claim 1, characterized in that, The generation of multi-level feature maps includes: The training dataset is input into the path aggregation network; The path aggregation network outputs a first feature map and a second feature map. The first feature map is a low-semantic high-resolution feature map used for small target reasoning, and the second feature map is a high-semantic low-resolution feature map used for ordinary target reasoning.
4. The smoking hazard identification method according to claim 3, characterized in that, The sparse anchor point screening process includes: Based on the second feature map, a sparse probability map is generated, wherein the value of each pixel in the sparse probability map represents the probability that a small target exists at the corresponding location; Filter the anchor points corresponding to pixels whose values in the sparse probability map satisfy the preset small target existence probability threshold, and use them as positive sample anchor points for small targets. For the dense branch anchor points corresponding to the first feature map, determine whether they simultaneously meet the preset conditions, filter the anchor points that meet the preset conditions as ordinary target positive sample anchor points, and summarize the small target positive sample anchor points and the ordinary target positive sample anchor points to obtain the filtered positive sample anchor point set.
5. The smoking hazard identification method according to claim 4, characterized in that, The preset conditions include: The anchor point center is located within a preset pixel range of the actual bounding box; The alignment index between the anchor point and the actual bounding box ranks among the top N preset values in all anchor boxes; The intersection-union ratio between the anchor point and the actual bounding box is the maximum.
6. The smoking hazard identification method according to claim 4, characterized in that, The completed model training, resulting in the initial smoking hazard identification model, includes: The training dataset is divided into a training subset and a validation subset according to a preset ratio; Using the pre-trained weights obtained from training on a general object detection dataset as initial weights, the training subset and the set of positive sample anchor points are input into the deep learning model to train the backbone network and prediction head of the model. The validation subset is input into the trained model to evaluate the model performance and adjust the hyperparameters until the model performance reaches a preset performance threshold. The adjusted model is then used as the initial smoking hazard identification model.
7. The smoking hazard identification method according to claim 1, characterized in that, The periodic collection of real-time smoking behavior images from industrial sites as incremental data includes: Set a preset time interval for data collection, and perform data collection operations according to the preset time interval; Smoking behavior images are captured in real time using monitoring equipment at the industrial site. The images include at least those taken under supplementary lighting during night shifts. All captured images are uploaded to the server as incremental data.
8. The smoking hazard identification method according to claim 1, characterized in that, The lightweight fine-tuning of the small target detection head of the initial smoking hazard identification model includes: Fix the backbone network parameters of the initial smoking hazard identification model; The iterative dataset is input into the initial smoking hazard identification model, and only the small target detection head parameters of the model are adjusted. The small target detection head is a convolutional module. Set a fine-tuning learning rate, with the initial learning rate set to a preset initial learning rate. After each preset training round, reduce the learning rate by a preset ratio and continue training until the model loss value is stable and less than a preset loss threshold to obtain the updated smoking hazard identification model.
9. A smoking hazard identification device, characterized in that, The device includes: Preprocessing module: Acquires and preprocesses multi-source image data to obtain a training dataset. The training dataset is then input into a deep learning model containing a path aggregation network. By generating multi-level feature maps and performing sparse anchor point filtering, the model training is completed, and an initial smoking hazard identification model is obtained. Lightweight fine-tuning module: Periodically collect real-time smoking behavior images as incremental data, fuse the incremental data with the training dataset to obtain an iterative dataset, and perform lightweight fine-tuning on the small target detection head of the initial smoking hazard identification model based on the iterative dataset to obtain an updated smoking hazard identification model; Inference Analysis Module: Input the real-time image stream from the industrial site into the updated smoking hazard identification model, perform inference analysis, and output the smoking hazard identification results.
10. A smoking hazard identification system, characterized in that, The system includes the smoking hazard identification device as described in claim 9.