Improved YOLO-based plant station illegal behavior monitoring method and system

Abstract

The invention provides a plant station illegal behavior intelligent monitoring method and system based on an improved YOLO algorithm, and belongs to the technical field of industrial plant station safety management. The monitoring method comprises the steps of video stream acquisition and preprocessing, improved YOLO feature extraction and detection, specific illegal behavior identification, illegal behavior decision, alarm linkage, platform interaction and the like. The core improvement lies in that a lightweight backbone network is constructed by adopting GhostNet, and model parameters and calculation amount are reduced; a small target detection effect is optimized by newly adding a high-resolution detection head, introducing a BiFPN fusion mechanism and adopting a Focus loss and attention mechanism; color band channel enhancement, time sequence difference features and optical flow detection are fused, and flame / smoke recognition robustness is improved; and multiple types of illegal behavior recognition are integrated in the same model system, so that unified monitoring is realized. According to the method, the problems of insufficient model lightweight, unstable small target detection, unreliable fire identification, complex deployment and the like in the prior art are solved.

Description

A Method and System for Monitoring Plant Violations Based on Improved YOLO Technical Field

[0001] This invention belongs to the field of behavior monitoring technology, and specifically relates to a method and system for monitoring plant violations based on an improved YOLO. Background Technology

[0002] Currently, plant safety supervision mainly relies on manual inspections or ordinary video surveillance. Traditional manual inspections are limited by manpower, making it difficult to achieve 24 / 7 uninterrupted supervision. Furthermore, human judgment is prone to subjectivity, missed detections, and slow response times. While ordinary video surveillance can record, it lacks AI intelligent analysis capabilities and cannot automatically identify violations. It still requires monitoring personnel to constantly monitor the screen, resulting in low efficiency.

[0003] While deep learning-based behavior recognition methods are beginning to be applied in the field of security monitoring, they still have the following shortcomings:

[0004] The models are not lightweight enough to meet the real-time deployment requirements at the edge. Many existing models have a large number of parameters and high computational overhead, making it difficult to run in real time on embedded devices and edge computing boxes on the factory site, resulting in high latency and high power consumption. They also have poor adaptability to diverse scenarios, such as varying factory lighting, complex camera angles, and concealed human behavior, leading to unstable detection of small targets like smoking and phone calls, resulting in false positives and false negatives. Flame / smoke recognition methods are simplistic and lack composite feature fusion. Some solutions rely solely on color or texture features, which are unsuitable for high-dynamic or low-light scenarios, leading to unreliable fire detection. The identification processes for different violations are not integrated, lacking a unified intelligent monitoring platform. Most solutions use independent algorithms, resulting in complex deployment and wasted computing power.

[0005] Therefore, an improved YOLO algorithm system is needed that is lightweight, high-precision, edge-deployable, and capable of unified monitoring of various violations such as smoking, phone calls, flames, and smoke, in order to achieve intelligent, secure, and real-time monitoring of violations at the plant. Summary of the Invention

[0006] In view of this, the present invention proposes a monitoring method and system for plant violations based on improved YOLO. Through network structure optimization, small target feature enhancement, fusion of highly robust flame / smoke recognition modules, and efficient edge deployment strategy, it can achieve real-time identification of dangerous behaviors such as smoking, making and receiving phone calls, smoke, and open flames.

[0007] The technical solution of this invention includes: a method for monitoring plant violations based on an improved YOLO, comprising:

[0008] S1: The real-time video stream of the plant monitoring area is acquired through the video acquisition module, and the video frames are preprocessed, including noise reduction, image normalization and brightness enhancement.

[0009] S2: The preprocessed video frames are input into the improved YOLO model deployed in the edge intelligent analysis unit for target detection. The improved YOLO model includes the following improvements:

[0010] The convolutional layers of the original YOLO model are replaced with GhostNet or CSPNet structures to construct a lightweight backbone network;

[0011] A high-resolution detection head is built at the feature output end of the lightweight backbone network, and a BiFPN feature fusion mechanism is introduced.

[0012] An attention mechanism is embedded to enhance the features of the channels where small targets such as cigarette butts and mobile phones are located;

[0013] A color band channel enhancement module for flame detection is introduced, and dynamic smoke texture analysis is performed by combining temporal difference features and optical flow information.

[0014] S3: Based on the detection results of step S2, smoking behavior is identified, specifically including: simultaneously detecting face / body, smoke and hand targets, and determining whether smoking behavior exists through key point association and spatial location relationship map;

[0015] Step S4: Based on the detection results of step S2, perform call answering behavior recognition, specifically including: detecting face / head, mobile phone and hand targets, analyzing the spatial relationship between the hand and the mobile phone, and using a continuous multi-frame voting mechanism to determine whether the call is in progress;

[0016] Step S5: Based on the detection results of step S2, flame and smoke detection is performed, specifically including: integrating YOLO static appearance feature detection, temporal frame difference analysis and the color band channel enhancement module for flame detection to make a comprehensive judgment;

[0017] Step S6: Perform multi-strategy fusion decision-making on the various violations identified in steps S3, S4, and S5. The strategies include confidence weighting, spatial constraint rules, and continuous frame judgment mechanism, and finally output the violation type.

[0018] Step S7: If a violation is detected, a local alarm is triggered, and the violation event information is uploaded to the management platform.

[0019] Preferably, the lightweight backbone network in S2 uses a Ghost module, and the ratio of the total parameters of its generated feature maps to the parameters of a standard convolution satisfies the following:

[0020] m main feature maps are generated using point convolution or narrow convolution;

[0021] Based on lightweight operations, additional features are linearly generated from each main feature map, resulting in a final output dimension of C. out The feature set;

[0022] Generate main feature parameters: Let the number of principal features m = αC out Where 0 < α < 1, the parameters required for the remaining channels are obtained by using lightweight operations to expand the range. ;

[0023] Total parameters:

[0024] Total parameter ratio:

[0025] ;

[0026] C out α is the number of output channels, k is the kernel size, and α is the scaling factor.

[0027] Preferably, the high-resolution detection head is added and multi-scale feature fusion is introduced, using the Focal Loss function as the classification loss function, and its gradient is calculated as follows:

[0028] Focal loss gradient:

[0029]

[0030] p is the model's predicted probability, and γ is an adjustment factor used to reduce the weight of easily classified samples and focus training on difficult-to-classify samples.

[0031] The first item It is the scaled CE gradient; in the second term As an additional term, it is small or negligible when 𝑝 is close to 1; when When the factor is close to 1, it is easy to classify positive samples. The gradient is significantly reduced; when p is small or close to 0, or for difficult samples, this factor is larger, and the gradient weight is amplified.

[0032] Preferably, in S2, an attention mechanism is introduced to enhance the features of small targets, and the signal-to-noise ratio of small targets is improved through multi-scale fusion; the fused signal-to-noise ratio SNR_fused is defined as:

[0033]

[0034] Assuming the noise is independent and has a mean of 0, the fused SNR is: ,choose , This allows the numerator to be enlarged while the denominator to be controlled to be smaller.

[0035] Preferably, the flame and smoke detection uses the DIoU loss function for bounding box regression, and the loss L_DIoU is defined as:

[0036]

[0037] Where b c g c The coordinates of the predicted bounding box and the center bounding box are real coordinates;

[0038] represents the Euclidean distance, and c represents the length of the diagonal line enclosing the two frames;

[0039] The gradient with respect to the center coordinates consists of two terms:

[0040] From Gradient: Drives changes in the shape / position of the border to increase the overlap area;

[0041] From distance The gradient: directly penalizes the center offset in a linear / quadratic form.

[0042] Preferably, the temporal frame difference analysis is used for feature recognition of smoke dynamic patterns:

[0043] Assuming that fire / smoke detection is performed based on static appearance color / texture in each frame, there will be false positive events. The false alarm probability of a single frame is P0, which is caused by background disturbance and changing lighting.

[0044] Temporal differential-inter-frame brightness / texture dynamic detection and optical flow-motion features are introduced as additional criteria: if the static appearance is determined to be fire / smoke, but the temporal and optical flow do not support it, it can be rejected. The false alarm probability in the absence of fire background is as follows:

[0045] It is a reduction relative to single evidence P0, with a ratio of q0.

[0046] Preferably, in step S4, "face / head," "phone," and "hand" are detected, and the spatial relationship between the hand and the phone determines whether the person is in a call. Continuous frame voting is used to reduce the probability bound of false alarms. If the frames are independent, the false alarm probability of k consecutive frames is p. k :

[0047] Combining the cumulative probability of the binomial distribution, we get:

[0048] Setting: The false positive probability of a single-frame detector for a certain type of violation is P, which is the probability of alarming in each frame when there is no violation;

[0049] The strategy of "only alarming when all k consecutive frames are positive" or "only alarming when ≥m frames within a sliding window are positive" is adopted, and it is assumed that false alarms in different frames are approximately independent.

[0050] Quantitative proof of reducing false alarm probability through consecutive frame judgment: If frames are independent, the false alarm probability of k consecutive frames is pk. The "≥m frames within a window" strategy can be given by the cumulative probability of a binomial distribution.

[0051] .

[0052] Furthermore, this invention also provides an intelligent monitoring system for plant violations based on an improved YOLO algorithm, used to implement the above method, including:

[0053] The video acquisition module is used to acquire video images of the plant monitoring area;

[0054] The edge intelligent analysis unit is equipped with an improved YOLO model for real-time target detection and behavior analysis of input video frames;

[0055] The violation identification module, connected to the edge intelligent analysis unit, is used to identify behaviors such as smoking, making or receiving phone calls, flames, and smoke based on the detection results.

[0056] A multi-category fusion decision module, connected to the violation identification module, is used to logically fuse and comprehensively decide on the identification results of multiple violations.

[0057] The alarm and push module is used to trigger local alarms and push event information to the management platform when a violation is detected.

[0058] The data management and logging module is used to store images, videos, and timestamp information related to alarm events.

[0059] Specifically, the edge intelligent analysis unit is an embedded device or edge computing box deployed at the factory site.

[0060] The plant violation monitoring method and system based on improved YOLO proposed in this invention have the following significant advantages compared with the prior art:

[0061] It enables unified identification of multiple types of violations, integrating smoking, phone calls, smoke, and flame detection into the same model system, making deployment simpler and less costly.

[0062] The ability to recognize small targets has been significantly improved, making it suitable for complex small target scenarios such as cigarette butts and mobile phones, and reducing the false detection rate by more than 40%.

[0063] It can run in real time at the edge, and the improved model parameters are reduced by 30% to 60%, increasing the FPS to 25 to 40 frames per second on embedded edge devices.

[0064] The robustness of flame / smoke recognition is improved by using color enhancement, temporal feature fusion, and optical flow detection to reduce the false alarm rate of fires by 30%.

[0065] It achieves stable monitoring around the clock and has excellent recognition capabilities for changes in lighting, backlighting, and nighttime scenes.

[0066] It can be widely used in various scenarios such as gas stations, chemical plants, power stations, oil and gas depots, and tunnels, and has significant economic and social benefits for safe production. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 is a schematic diagram of the overall system architecture provided by the present invention;

[0069] Figure 2 is a schematic diagram of the improved YOLO network architecture provided by the present invention;

[0070] Figure 3 is a flowchart of the smoking behavior recognition process provided by the present invention;

[0071] Figure 4 is a flowchart of the telephone call behavior recognition provided by the present invention;

[0072] Figure 5 is a flowchart of the flame / smoke recognition process provided by the present invention;

[0073] Figure 6 is a schematic diagram of edge deployment and alarm linkage provided by the present invention. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, to avoid obscuring the invention with unnecessary details, only processing steps closely related to the solution of this invention are shown in the drawings, while other details not closely related to this invention are omitted.

[0075] To address the shortcomings of existing deep learning-based behavior recognition methods in security monitoring, such as insufficient model lightweighting, poor adaptability to diverse scenarios, and limited flame / smoke recognition methods, this invention provides a plant violation monitoring method based on an improved YOLO, comprising the following steps:

[0076] S1: Video stream acquisition and preprocessing: Real-time video from the camera is captured in frames, denoised, normalized, and brightened to improve the quality of the model input.

[0077] S2: Improved YOLO model. As shown in Figure 2, the improved YOLO model has been optimized in the following aspects:

[0078] 1) Lightweight backbone network: GhostNet is used to replace some standard convolutional layers, reducing the number of parameters and computation. In the specific implementation, GhostNet and other structures are used to replace some convolutions, reducing the number of parameters and improving the speed at the edge.

[0079] Small object detection optimization, Ghost approach: First, generate m main feature maps using a small pointwise convolution or narrow convolution. Then, use cheap operations (channel-wise linear transformation / depthiably separable small convolution / linear combination) to linearly generate additional features from each main feature map, finally obtaining C. out One characteristic.

[0080] The parameters for generating the main features are approximately Let the number of features be m = αC out , where 0 < α < 1, (usually 0.2-0.5);

[0081] Extend the remaining channels using the cheap operation with the required parameters. Where s is the unit parameter of the cheap operation (much smaller than k) 2 C in (on the order of magnitude)

[0082] Total parameters:

[0083] Overall parameter ratio (Ghost / Standard):

[0084]

[0085] If C in Approximately C out And s is much smaller than k 2 C in If ρ is approximately equal to α, then the parameter reduction factor is approximately α.

[0086] Engineering Explanation: The Ghost architecture can significantly reduce model parameters and computational load, thereby improving FPS and energy efficiency on edge devices.

[0087] 2) New high-resolution detection head: Replacing cross-entropy with Focal Loss (parameter γ>0) will significantly reduce the gradient weight for easily classified samples, thus paying more attention to hard samples when the classes are imbalanced (there are far more negative samples than positive samples), which is beneficial for learning small targets / rare violations.

[0088] Focal loss gradient:

[0089]

[0090] Let p be the predicted probability of the model for the positive class; the first term is... The scaled CE gradient; the second term is an extra term (usually positive), mainly in... When close to 1, it is small or negligible. Key observation: When When the factor is close to 1 (easily classified positive samples), When p is small, the gradient is significantly reduced; when p is small, the factor is large, and the gradient weight is amplified.

[0091] Focal loss is achieved through The weights are adjusted adaptively to shift the training focus from a large number of easily classified negative / positive samples to difficult samples, thereby improving the learning performance of rare behaviors (such as smoking and making phone calls in small samples).

[0092] 3) The gradient of the loss with respect to the convergence of localization is set as follows:

[0093] Let the shallow features be represented as vector x s The deep feature is x d For small targets, the shallow layer contains high-frequency signals s and noise n. s :x s =s+n. A deeper layer provides a greater semantic response. s (a is the amplification factor) and less noise n d :x d =a s +n d (However, at low resolutions, details may be lost.) A simple merging is achieved using a weighted sum:

[0094]

[0095] The signal-to-noise ratio (SNR) is defined as follows: Assuming the noise is independent and has a mean of 0, the fused SNR is: .choose , This allows for the amplification of the numerator while keeping the denominator relatively small.

[0096] Multi-scale fusion improves the detection probability by compensating for semantic weaknesses at a shallow level and enhancing semantic responses at a deep level, thereby amplifying small target signals and suppressing noise. BiFPN is a weighted learning fusion processor that can automatically find an approximately optimal solution set.

[0097] 4) The introduction of color band channel enhancement (Fire-Color Enhancement) is demonstrated as follows:

[0098] Loss definition: Compared to traditional L2 regression, which only performs point-to-point regression on coordinate differences (without directly considering the overlapping area), the gradient direction of DIoU is closer to the final optimization goal (max IoU). Therefore, in the early stages of training, DIoU will provide a direction that is more conducive to improving IoU rather than just reducing coordinate errors, which can usually speed up convergence based on IoU.

[0099] Using DIoU / CIoU loss can make the model converge to a solution with high IoU accuracy faster, reducing positioning offset and false alarms.

[0100] Identifying dynamic smoke patterns using temporal difference features:

[0101] Assuming that fire / smoke detection is performed on each frame based solely on static appearance (color / texture), there will be false positive events. The false alarm probability per frame is P0, which is caused by background disturbances, changing lighting, etc.

[0102] Temporal differential (inter-frame brightness / texture dynamic detection) and optical flow (motion features) are introduced as additional criteria: if a static appearance indicates fire / smoke but temporal and optical flow do not support it, the detection can be rejected. False alarm probability in a no-fire background:

[0103] prove:

[0104] Let event A represent "static detection positive judgment" (the probability is approximately P(A) = s, which is higher under real fire / smoke conditions, but may be equal to P0 in the absence of fire).

[0105] Let event B represent “time-motion evidence support” (probability q1 under real fire / smoke, probability q0 under no-fire background), generally q1≫q0;

[0106] The determination is made using a combination of two pieces of evidence: fire / smoke is determined if and only if A ∧ B.

[0107] This is a reduction relative to single evidence P0, with a ratio of approximately q0, and all false alarms are significantly reduced.

[0108] The combined determination of temporal and optical flow motion characteristics with static appearance characteristics can multiplicatively reduce the false alarm rate in no-fire conditions, thereby improving robustness. Fusion optical flow information is used to distinguish between disturbances and real smoke.

[0109] S3: Smoking behavior recognition, as shown in Figure 3, includes the following steps:

[0110] Detects three types of targets in an image: faces / body, hands, and cigarette butts.

[0111] A spatial relationship diagram is established by associating key points (such as wrist and facial key points).

[0112] The system determines whether a hand is holding a cigarette and whether the cigarette butt is close to the mouth. If these conditions are met for multiple frames (e.g., more than 5 consecutive frames), it is considered a smoking behavior.

[0113] S4: As shown in Figure 4, the call answering / receiving behavior recognition process is as follows:

[0114] It detects three types of targets in an image: face / head, hand, and mobile phone.

[0115] Determine whether the hand is holding the phone and whether the phone is close to the ear area.

[0116] Using consecutive frame voting reduces the probability bound of false alarms. If the frames are independent, the probability of a false alarm per k consecutive frames is p. k Therefore, combining the cumulative probability of the binomial distribution, we get:

[0117]

[0118] Continuous confirmation significantly reduces single-frame jitter-type and sporadic false alarms and improves the reliability of alarms, but it increases response delay and sensitivity to short-term real events. Quantitative proof of reducing the probability of false alarms through continuous frame judgment.

[0119] S5: Flame / Smoke Recognition: As shown in Figure 5, flame and smoke recognition employs a multi-feature fusion strategy.

[0120] Static Feature Detection: Detecting static appearance features of flames and smoke using an improved YOLO model.

[0121] Temporal difference analysis: Calculates the brightness difference and texture changes between consecutive frames to extract dynamic features.

[0122] Color probability model: A probability model is established based on the color distribution of flames in the RGB / HSV space to enhance the ability to distinguish flame areas.

[0123] Optical flow motion analysis: Combine dense optical flow algorithm to determine the movement trend of flames / smoke and distinguish between real fire and interference objects (such as light reflection, moving objects).

[0124] Finally, by weighted fusion of the above multi-feature results, the flame / smoke identification conclusion is output.

[0125] S6: Violation Decision-Making: A multi-strategy fusion decision-making approach is adopted, including confidence weighting, spatial constraint rules, and a continuous frame judgment mechanism. The detection results of each violation identification submodule are weighted by confidence, assigning corresponding weights based on the confidence level of different detection results; spatial constraint rules are used to exclude detection results that do not conform to actual spatial logic; and the continuous frame judgment mechanism further reduces false detections. Considering all these factors, the violation type is ultimately determined, including four categories: smoking, making phone calls, flames, and smoke.

[0126] S7: Alarm Linkage and Platform Interaction: When a violation is detected by S6, the edge intelligent analysis unit sends an alarm signal to the alarm and push module. The alarm and push module triggers a local audible and visual alarm, promptly alerting on-site personnel to safety. Simultaneously, it uploads relevant data such as images and video clips of the violation to the cloud platform and pushes notifications to management personnel's mobile terminals or the central control platform, enabling management to be promptly aware of the violation and take appropriate action. The data management and log recording module synchronously saves alarm images, timestamps, video clips, and other data for subsequent querying and tracing.

[0127] Based on the above method, the present invention also provides the following system: consisting of a video acquisition module, an edge intelligent analysis unit, a violation behavior identification module, a multi-category fusion judgment module, an alarm and push module, and a data management and log recording module. The modules work together to realize intelligent monitoring and early warning of violations in the plant.

[0128] Video acquisition module: Using existing cameras in the plant or adding new high-definition cameras, it is used to collect video stream data from the plant's production site in real time, providing raw data support for subsequent analysis.

[0129] Edge Intelligent Analysis Unit: Deploys an improved YOLO model and is responsible for performing detection and inference on the video data acquired by the video acquisition module. It is the core unit for the system to achieve intelligent recognition.

[0130] The violation detection module includes four sub-modules: smoking detection, phone call detection, smoke detection, and flame detection, which are used to identify corresponding violations and dangerous scenarios.

[0131] Multi-category fusion judgment module: Logically fuses the detection results of the violation recognition module, makes a comprehensive judgment, improves the recognition accuracy, and reduces false detections and false negatives.

[0132] Alarm and push module: When a violation is detected, a local audible and visual alarm is triggered, and relevant information about the violation is pushed to the central control platform to promptly remind managers to take action.

[0133] Data management and logging module: responsible for saving alarm images, timestamps of events, related video clips, and other data for easy querying, tracing, and analysis.

[0134] Example 1

[0135] Hardware deployment:

[0136] This system was deployed at a gas plant. The video acquisition module uses 10 high-definition network cameras, installed in key locations such as the production area, storage area, and office area. The cameras support 1080P resolution and a frame rate of 30 frames per second to ensure the clarity and real-time performance of the video capture. The edge intelligent analysis unit uses an embedded edge computing box with the following configuration: a quad-core ARM Cortex-A53 processor with a main frequency of 1.5GHz, a Mali-T860MP4 GPU, 4GB of memory, and 64GB of storage capacity, meeting the requirements for running the improved YOLO model. The alarm and push module uses audible and visual alarms and an SMS push module. The audible and visual alarms are installed in prominent locations in each area, and the SMS push module is linked to the mobile phones of management personnel.

[0137] Model training and deployment:

[0138] Image data of violations and dangerous scenarios such as smoking, making phone calls, smoke, and flames were collected at the factory site to construct a dataset of 100,000 images, including 80,000 images for training, 10,000 images for validation, and 10,000 images for testing. The dataset was labeled with object categories (face / body, smoke, hand, mobile phone, smoke, flame) and object bounding boxes. Based on the PyTorch deep learning framework, a modified YOLO algorithm was used for model training. The mini-batch stochastic gradient descent (SGD) optimizer was employed during training, with an initial learning rate of 0.001 that gradually decreased with each training epoch. After training, the model was deployed to edge computing boxes, resulting in a 45% reduction in model parameters compared to the traditional YOLO model.

[0139] System operation and effect verification

[0140] After the system starts, the video acquisition module collects video streams from various areas in real time, and transmits them to the edge intelligent analysis unit after preprocessing. The improved YOLO model in the edge intelligent analysis unit performs feature extraction and detection on the preprocessed image, and makes a decision through the multi-class fusion decision module.

[0141] When smoking is detected, the system triggers a local audible and visual alarm within 0.3 seconds, simultaneously uploading an alarm image, timestamp, and relevant video clips to the cloud platform and sending an SMS notification to administrators. Administrators can view detailed information through the central control platform. Testing showed that the accuracy rate for identifying smoking behavior reached 92%, with a 42% reduction in the false detection rate.

[0142] For phone call behavior, the system judges by voting through 5 consecutive frames, with a false alarm rate controlled within 3% and an identification response time of 0.5 seconds.

[0143] In flame / smoke detection tests, the system accurately identified fires under different lighting conditions (strong light, weak light, backlight) and different scenarios (indoor, outdoor, enclosed space), reducing the false alarm rate by 35% and achieving a response time as fast as 0.2 seconds.

[0144] The entire system runs at a frame rate of 32 frames per second on the edge computing box, meeting the requirements for real-time monitoring, and its energy consumption is reduced by 38% compared to the traditional model.

[0145] Based on actual testing, this system performs excellently in the following aspects:

[0146] Detection accuracy: On the self-built factory site violation dataset, the mAP (mean accuracy) of the four types of behaviors, namely smoking, making phone calls, flames, and smoke, reached 92.5%, which is about 8% higher than the baseline YOLOv5.

[0147] Inference speed: On the Jetson Nano edge device, the number of model parameters is reduced by about 45%, and the inference speed reaches 32 FPS, which meets the requirements of real-time monitoring.

[0148] False alarm rate: Through multi-feature fusion and continuous frame judgment mechanism, the false alarm rate of smoking and phone call behavior is reduced by more than 40%, and the false alarm rate of flame and smoke is reduced by 30%.

[0149] Environmental adaptability: It maintains high recognition stability in complex scenarios such as low light, backlight, and nighttime, enabling 24 / 7 all-weather monitoring.

[0150] This system has been successfully applied in actual monitoring scenarios of multiple gas plants, chemical plants, and power substations, effectively improving the level of intelligent safety management of the plants and reducing the cost of manual inspections, resulting in significant economic and social benefits.

[0151] The above description is merely a preferred embodiment of the present invention and is not intended to limit the patent scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for monitoring plant violations based on improved YOLO, characterized in that, include: S1: The real-time video stream of the plant monitoring area is acquired through the video acquisition module, and the video frames are preprocessed, including noise reduction, image normalization and brightness enhancement. S2: Input the preprocessed video frames into the improved YOLO model deployed in the edge intelligent analysis unit for target detection. The improved YOLO model includes the following improvements: replacing the convolutional layers of the original YOLO model with a GhostNet structure to construct a lightweight backbone network. At the feature output end of the lightweight backbone network, a high-resolution detection head is built, and a BiFPN feature fusion mechanism is introduced; an attention mechanism is embedded to enhance the features of the channels where small targets such as cigarette butts and mobile phones are located; a color band channel enhancement module for flame detection is introduced, and dynamic texture analysis of smoke is performed by combining temporal difference features and optical flow information; S3: Based on the detection results of step S2, smoking behavior recognition is performed, specifically including: simultaneously detecting face / body, smoke and hand targets, and determining whether smoking behavior exists through key point association and spatial position relationship map; Step S4: Based on the detection results of step S2, phone call behavior recognition is performed, specifically including: detecting face / head, mobile phone and hand. The system aims to determine whether a call is in progress by analyzing the spatial relationship between the hand and the phone and using a continuous multi-frame voting mechanism. Step S5: Based on the detection results of step S2, flame and smoke detection is performed, specifically including: fusing YOLO static appearance feature detection, temporal frame difference analysis, and the color band channel enhancement module for flame detection for comprehensive judgment. Step S6: A multi-strategy fusion decision is made for various violations identified in steps S3, S4, and S5. The strategies include confidence weighting, spatial constraint rules, and continuous frame judgment mechanism, and finally, the violation type is output. Step S7: If a violation is determined to exist, a local alarm is triggered, and the violation event information is uploaded to the management platform.

2. The method for monitoring plant violations based on improved YOLO according to claim 1, characterized in that, The lightweight backbone network described in S2 uses the Ghost module, whose total parameters in the generated feature maps are proportional to the parameters of a standard convolution, satisfying the following: m main feature maps are generated using pointwise convolutions or narrow convolutions; additional features are linearly generated from each main feature map based on lightweight operations, resulting in a final output dimension of C. out The feature set; Generate main feature parameters: Let the number of principal features m = αC out Where 0 < α < 1, the parameters required for the remaining channels are obtained by using lightweight operations to expand the range. Total parameters: Total parameter ratio: C out α is the number of output channels, k is the kernel size, and α is the scaling factor.

3. The method for monitoring plant violations based on improved YOLO according to claim 1, characterized in that, The high-resolution detection head is added, and multi-scale feature fusion is introduced. The Focal Loss function is used as the classification loss function, and its gradient is calculated as follows: Focal loss gradient: p represents the model's predicted probability, and γ is a regulation factor used to reduce the weight of easily classified samples and focus training on difficult-to-classify samples; in the first term... It is the scaled CE gradient; in the second term As an additional term, it is small or negligible when 𝑝 is close to 1; when When the factor is close to 1, it is easy to classify positive samples. The gradient is significantly reduced; when p is small or close to 0, or for difficult samples, this factor is larger, and the gradient weight is amplified.

4. The method for monitoring plant violations based on improved YOLO according to claim 1, characterized in that, In S2, an attention mechanism is introduced to enhance the features of small targets, and the signal-to-noise ratio (SNR) of small targets is improved through multi-scale fusion; the fused SNR_fused is defined as: Assuming the noise is independent and has a mean of 0, the fused SNR is: ,choose ， This allows the numerator to be enlarged while the denominator to be controlled to be smaller.

5. The method for monitoring plant violations based on improved YOLO according to claim 1, characterized in that, The flame and smoke detection uses the DIoU loss function for bounding box regression, and the loss L_DIoU is defined as: Where b c g c The coordinates of the predicted bounding box and the center bounding box are real coordinates; The gradient represents the Euclidean distance, where c represents the length of the diagonal line enclosing the two bounding boxes; the gradient with respect to the center coordinates includes two terms: from Gradient: Drives changes in the shape / position of the border to increase the overlapping area; derived from distance The gradient: directly penalizes the center offset in a linear / quadratic form.

6. The method for monitoring plant violations based on improved YOLO according to claim 1, characterized in that, The temporal frame difference analysis is used for feature recognition of dynamic smoke textures: It is assumed that detecting fire / smoke based solely on static appearance color / texture in each frame will result in false positives, with a single-frame false alarm probability of P0 caused by background disturbances and changing illumination. Temporal difference-inter-frame brightness / texture dynamic detection and optical flow-motion features are introduced as additional criteria: if the static appearance is determined to be fire / smoke, but the temporal and optical flow do not support it, it can be rejected. The false alarm probability in a no-fire background is: It is a reduction relative to single evidence P0, with a ratio of q0.

7. The method for monitoring plant violations based on improved YOLO according to claim 1, characterized in that, S4 detects "face / head", "phone", and "hand", and determines whether a call is in progress based on the spatial relationship between the hand and the phone. It uses consecutive frame voting to reduce the probability bound of false alarms; if the frames are independent, the false alarm probability for k consecutive frames is p. k Combining the binomial cumulative probability, we obtain: Let P be the false positive probability of a single-frame detector for a certain type of violation, i.e., the probability of an alarm per frame when there is no violation; adopt a strategy of "alarm only when all k consecutive frames are positive" or "positive alarm only when ≥m frames within a sliding window", and assume that false alarms in different frames are approximately independent; quantify the reduction of false alarm probability through consecutive frame judgment: if frames are independent, then the probability of false alarms in all k consecutive frames is p. k The "≥m frames within a window" strategy can be given by the cumulative probability of a binomial distribution: 。 8. A plant violation intelligent monitoring system based on an improved YOLO algorithm, used to implement the method described in any one of claims 1-7, characterized in that, include: The video acquisition module is used to acquire video images of the plant monitoring area; The edge intelligent analysis unit is equipped with an improved YOLO model for real-time target detection and behavior analysis of input video frames; The violation identification module, connected to the edge intelligent analysis unit, is used to identify behaviors such as smoking, making or receiving phone calls, flames, and smoke based on the detection results. The multi-category fusion judgment module, connected to the violation identification module, is used to logically fuse and comprehensively decide on the identification results of multiple violations; the alarm and push module is used to trigger a local alarm and push event information to the management platform when a violation is determined. The data management and logging module is used to store images, videos, and timestamp information related to alarm events.

9. The system according to claim 8, characterized in that, The edge intelligent analysis unit is an embedded device or edge computing box deployed at the factory site.

Images