Early warning method and system for factory smoke based on deep learning

Abstract

This invention relates to the field of image processing technology, specifically to a method and system for early monitoring and warning of factory smoke based on deep learning. The method involves obtaining a comprehensive difference degree based on color difference features and neighborhood difference features at the same location in a video frame; obtaining a first pixel and a second pixel based on the comprehensive difference degree; performing corner matching between the second pixel and pixels at the same location in the previous video frame to obtain matching pairs and unmatched points; obtaining a learning rate adjustment parameter based on the comprehensive difference degree and coordinate distance features of the matching pairs; and obtaining a learning rate adjustment parameter for the unmatched points based on the comprehensive difference degree and coordinate distance features between the unmatched points and suspected matching points. This invention adjusts the preset learning rate based on the pixel learning rate adjustment parameter, and uses the MOG2 algorithm and adaptive learning rate to separate the foreground and background of the current video frame and monitor smoke, thereby improving the accuracy of smoke monitoring and warning.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for early monitoring and warning of factory smoke based on deep learning. Background Technology

[0002] In high-risk fields such as chemical, metallurgical, and electronics manufacturing, abnormal smoke is often an early sign of potential accidents such as equipment overheating, abnormal chemical reactions, and material combustion. Failure to detect and address these issues promptly can lead to a series of accidents, thus necessitating smoke monitoring. In the complex environment of factories, smoke exhibits characteristics such as diffusion and dynamic changes. Relying solely on a single sensor for monitoring often results in problems such as alarm delays, insufficient sensitivity, or high false alarm rates. To achieve more accurate early warning, intelligent fusion analysis can be performed by comprehensively utilizing smoke video image features and sensor data. However, background interference in factory settings, such as changes in lighting and the movement of personnel and equipment, can affect the model's accurate extraction of the smoke foreground, reducing the accuracy of identification and warning.

[0003] The existing MOG2 Gaussian mixture model separation algorithm can separate foreground and background. The learning rate in this algorithm controls the update speed of the background model, thus balancing its adaptability to environmental changes and its sensitivity to foreground targets. If the learning rate is set too high, the model updates too quickly, easily misclassifying parts that should be estimated as background as foreground. That is, some stable or regularly changing background elements are classified as foreground, leading to oversensitivity and misjudgment. If the learning rate is set too low, the model updates too slowly, failing to make timely and sensitive estimations of dynamic regions. It may still misclassify rapidly changing or irregularly changing objects as background, resulting in low sensitivity and insufficient foreground identification. Therefore, a fixed learning rate is insufficient to simultaneously balance model stability and sensitivity in identifying smoke changes, leading to poor foreground and background separation and affecting the accuracy of smoke feature extraction and early smoke detection. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention aims to provide a method and system for early monitoring and warning of factory smoke based on deep learning. The specific technical solution adopted is as follows:

[0005] Acquire video frames from factory monitoring;

[0006] A color difference feature value is obtained based on the color difference features at the same position between the current video frame and the previous video frame; a comprehensive difference degree is obtained based on the neighborhood difference features at the same position between the current video frame and the previous video frame and the color difference feature value; and a first pixel and a second pixel in the current video frame are obtained based on the comprehensive difference degree.

[0007] Corner point matching is performed between the second pixel and pixels at the same position in the previous video frame to obtain different matching pairs and unmatched points in the second pixel; the learning rate adjustment parameters of the matching pairs are obtained based on the comprehensive difference degree and coordinate distance features of the matching pairs; suspected matching points are obtained based on the relative coordinates of the unmatched points and different matching pairs; the learning rate adjustment parameters of the unmatched points are obtained based on the comprehensive difference degree and coordinate distance features between the unmatched points and the suspected matching points.

[0008] The preset learning rate is adjusted according to the learning rate adjustment parameters of different pixels in the current video frame to obtain the adaptive learning rate of different pixels; the foreground and background of the current video frame are separated according to the MOG2 algorithm and the adaptive learning rate; and smoke is monitored according to the separation results.

[0009] Furthermore, the step of obtaining color difference feature values ​​based on color difference features at the same position between the current video frame and the previous video frame includes:

[0010] In the formula, D represents the color difference characteristic value at the same position. This represents the A channel value of the LAB color model for pixels at the same position in the current video frame. This represents the A channel value of the LAB color model for pixels at the same position in the previous video frame. This represents the B channel value of the LAB color model for pixels at the same position in the current video frame. This represents the B channel value of the LAB color model for pixels at the same position in the previous video frame.

[0011] Furthermore, the step of obtaining the comprehensive difference degree based on the neighborhood difference features of the same position between the current video frame and the previous video frame, and the color difference feature value, includes:

[0012] In the formula, W represents the overall degree of difference at the same position. This represents the cosine similarity of feature description vectors at the same location obtained through the SIFT algorithm, where D represents the color difference feature value. This indicates linear normalization.

[0013] Further, the step of obtaining the first pixel and the second pixel in the current video frame based on the comprehensive difference degree includes:

[0014] The pixel in the current video frame whose overall difference is a constant of 0 is selected as the first pixel, and the other pixel is selected as the second pixel.

[0015] Further, the step of performing corner matching between the second pixel and pixels at the same position in the previous video frame to obtain different matching pairs and unmatched points in the second pixel includes:

[0016] The corner points in the second pixel at the same position as those in the previous video frame are obtained using the FATS corner detection algorithm; the feature description vectors of the corner points are obtained using the SIFT algorithm; the cosine similarity between the feature description vectors of the corner points in the current video frame and any corner point in the previous video frame is calculated; the corner point relationship corresponding to the maximum cosine similarity of each corner point in the current video frame is taken as the matching pair; and the non-corner points in the second pixel are taken as the unmatched points.

[0017] Furthermore, the step of obtaining the learning rate adjustment parameter of the matching pair based on the comprehensive difference degree and coordinate distance feature of the matching pair includes:

[0018] Calculate and normalize the straight-line distance between the coordinates of the matching pair to obtain the first distance; calculate the average of the comprehensive difference degree of the matching pair and the first distance to obtain the learning rate adjustment parameter of the matching pair.

[0019] Further, the step of obtaining suspected matching points based on the relative coordinates of the unmatched points and different matching pairs includes:

[0020] Using the corner points of the matched pair as the origin, a coordinate system is constructed based on the gradient direction of the corner points and the perpendicular line of the gradient direction. The perpendicular line of the gradient direction points to the side with the largest adjacent gray value on both sides of the gradient direction. The relative coordinates of the unmatched point in the coordinate system of any corner point in the current video frame are obtained, and the pixel point at the relative coordinate in the coordinate system of the previous video frame of the matched pair is taken as the suspected matching point of the unmatched point.

[0021] Further, the step of obtaining the learning rate adjustment parameter of the unmatched point based on the comprehensive difference degree and coordinate distance feature between the unmatched point and the suspected matching point includes:

[0022] Calculate the minimum overall difference between the unmatched point and all suspected matching points to obtain a first value; calculate and normalize the straight-line distance between the suspected matching point and the unmatched point corresponding to the first value to obtain a second distance; calculate the average of the first value and the second distance to obtain the learning rate adjustment parameter of the unmatched point.

[0023] Further, the step of adjusting the preset learning rate according to the learning rate adjustment parameters of different pixels in the current video frame to obtain an adaptive learning rate for different pixels includes:

[0024] Calculate the difference between the preset maximum learning rate and the preset minimum learning rate to obtain the adjustment benchmark; calculate the product of the adjustment benchmark and the learning rate adjustment parameter to obtain the adjustment amount; calculate the sum of the preset minimum learning rate and the adjustment amount to obtain the adaptive learning rate of different pixels in the current video frame; the learning rate adjustment parameter of the first pixel is a constant 0.

[0025] The present invention also proposes a deep learning-based early monitoring and warning system for factory smoke, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of any one of the deep learning-based methods for early monitoring and warning of factory smoke.

[0026] The present invention has the following beneficial effects:

[0027] In this invention, obtaining color difference feature values ​​allows for the analysis of pixel stability based on color difference features at phase positions in adjacent video frames. Obtaining comprehensive difference levels further analyzes pixel stability based on neighborhood difference features and color difference feature values ​​at the same position in adjacent video frames, thereby classifying pixels with different degrees of motion and initially determining the learning rate for different pixels. Obtaining matching pairs can be used to analyze the motion level of pixels, thus obtaining a more accurate learning rate. Obtaining the learning rate adjustment parameters for matching pairs allows for the determination of a suitable learning rate based on the comprehensive difference level and coordinate distance features of the matching pairs. Obtaining suspected matching points for unmatched points can determine the motion level of pixels with weak structural features in the video frame, and obtaining the learning rate adjustment parameters for unmatched points allows for the determination of a suitable learning rate based on the comprehensive difference level and coordinate distance features between the unmatched points and suspected matching points. Finally, the preset learning rate is adjusted according to the learning rate adjustment parameters of different pixels in the current video frame to obtain an adaptive learning rate for different pixels, thereby making the MOG2 algorithm more accurate in separating the foreground and background of factory videos, thus improving the accuracy of smoke detection and early warning. Attached Figure Description

[0028] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 The flowchart illustrates a deep learning-based method for early monitoring and warning of factory smoke, as provided in one embodiment of the present invention. Detailed Implementation

[0030] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a deep learning-based early monitoring and warning method and system for factory smoke based on the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0032] The following description, in conjunction with the accompanying drawings, details a specific scheme for an early monitoring and warning method and system for factory smoke based on deep learning, provided by this invention.

[0033] Please see Figure 1 The diagram illustrates a flowchart of a deep learning-based method and system for early monitoring and warning of factory smoke, according to an embodiment of the present invention. The method includes the following steps:

[0034] Step S1: Obtain video frames from factory monitoring.

[0035] In this embodiment of the invention, the implementation scenario is to conduct early monitoring and warning of factory smoke to improve monitoring accuracy; firstly, video frames from factory monitoring are acquired.

[0036] Step S2: Obtain color difference feature value based on color difference features at the same position between the current video frame and the previous video frame; obtain comprehensive difference degree based on neighborhood difference features and color difference feature value at the same position between the current video frame and the previous video frame; obtain the first pixel and the second pixel in the current video frame based on the comprehensive difference degree.

[0037] When separating smoke from the background in a video frame, the existing MOG2 Gaussian mixture model separation algorithm can be used. However, the fixed learning rate in this algorithm is difficult to simultaneously consider both background stability and the sensitivity to smoke changes. Therefore, the learning rate of each pixel needs to be adaptively adjusted. The learning rate of the current video frame is related to the motion and local stability of the pixels. In this embodiment of the invention, the stability characteristics of the pixels are analyzed from two dimensions: one is based on the color change of the pixel itself, and the other is based on the change of the neighborhood structure of the pixel. By using color and neighborhood structure information, the learning rate of different pixels can be accurately adjusted. First, the color difference feature value is obtained based on the color difference features at the same position between the current video frame and the previous video frame. Preferably, in this embodiment of the invention, the step of obtaining the color difference feature value includes:

[0038]

[0039] In the formula, D represents the color difference feature value at the same position. This represents the A channel value of the LAB color model for pixels at the same position in the current video frame. This represents the A channel value of the LAB color model for pixels at the same position in the previous video frame. This represents the B channel value of the LAB color model for pixels at the same position in the current video frame. This represents the B channel value of the LAB color model for pixels at the same location in the previous video frame. Converting video frames to the LAB color space and analyzing only the A and B chromaticity channels, discarding the L luminance channel, effectively avoids interference from brightness fluctuations caused by changes in ambient lighting on the calculation of differences between pixels at the same location. For example, even if the object's position remains unchanged, changes in overall lighting can still lead to differences in RGB or grayscale values. Directly subtracting based on these channels can easily cause misjudgments, thus affecting the accuracy of the learning rate estimation. The larger the color difference feature value, the more likely pixels at the same location in adjacent video frames are to represent different objects.

[0040] Furthermore, the overall difference level can be obtained based on the neighborhood difference features and color difference feature values ​​of the same position between the current video frame and the previous video frame; preferably, in this embodiment of the invention, the step of obtaining the overall difference level includes:

[0041]

[0042] In the formula, W represents the overall degree of difference at the same location. This represents the cosine similarity of feature description vectors at the same location obtained through the SIFT algorithm, where D represents the color difference feature value. This represents linear normalization. It should be noted that the SIFT scale-invariant feature transformation algorithm is an existing technology. It extracts feature description vectors from neighboring pixels within a certain range, centered on each pixel. The specific steps are not detailed here. A higher cosine similarity between feature description vectors at the same location indicates greater similarity in neighborhood information, making the pixel more stable. Therefore, a greater overall difference means a greater color difference and a greater difference in neighborhood structure between adjacent frames for that pixel, resulting in weaker local stability features for that pixel.

[0043] Further, after obtaining the overall difference level of all pixels in the current video frame, the first pixel and the second pixel in the current video frame can be obtained based on the overall difference level. Preferably, in this embodiment of the invention, the step of obtaining the first pixel and the second pixel includes: taking the pixel in the current video frame with an overall difference level of constant 0 as the first pixel, and otherwise taking it as the second pixel; the first pixel represents the pixel that has not changed between the current video frame and the previous video frame; the second pixel represents the pixel that may be smoke or moving objects. The greater the degree of motion and the worse the stability of the pixel, the greater its learning rate needs to be. For the first pixel that has not changed, its learning rate can be set to a smaller value, and the learning rate adjustment parameter can be set to 0.

[0044] Step S3: Perform corner matching on the second pixel and the pixels at the same position in the previous video frame to obtain different matching pairs and unmatched points in the second pixel; obtain the learning rate adjustment parameters of the matching pairs based on the comprehensive difference degree and coordinate distance features of the matching pairs; obtain the suspected matching points based on the relative coordinates of the unmatched points and different matching pairs; obtain the learning rate adjustment parameters of the unmatched points based on the comprehensive difference degree and coordinate distance features between the unmatched points and the suspected matching points.

[0045] The greater the motion and the worse the stability of a pixel, the higher its learning rate needs to be. To further obtain the motion features of objects in the current video frame, corner matching is first performed on the second pixel and the pixels at the same position in the previous video frame to obtain different matching pairs and unmatched points in the second pixel. Preferably, in this embodiment of the invention, the step of obtaining different matching pairs and unmatched points in the second pixel includes: obtaining the corner points at the same position in the previous video frame using the FATS corner detection algorithm. It should be noted that the FATS corner detection algorithm is existing technology, and the specific steps will not be described in detail. Corner points represent strong structural features in a video frame, so corner points can be prioritized for matching. The feature description vector of the corner point is obtained using the SIFT algorithm, and the cosine similarity between the feature description vector of the corner point in the current video frame and any corner point in the previous video frame is calculated. The corner point relationship corresponding to the maximum cosine similarity of each corner point in the current video frame is taken as the matching pair. When the cosine similarity of the feature description vectors of any two corner points in adjacent video frames is greater, it means that the neighborhood structure at the two corner points is more similar, and the two corner points are more likely to represent the same feature, thus increasing the probability of matching. After matching all corner points in the current video frame to obtain matching pairs, the non-corner points in the second pixel are taken as unmatched points.

[0046] Furthermore, the degree of motion can be determined based on the matched corner points. Therefore, the learning rate adjustment parameters of the matched pairs are obtained based on the comprehensive difference degree and coordinate distance features. Preferably, in this embodiment of the invention, the step of obtaining the learning rate adjustment parameters of the matched pairs includes: calculating and normalizing the straight-line distance between the coordinates of the matched pairs to obtain a first distance. It should be noted that, in order to avoid errors caused by the different actual captured image sizes of different video frames, the two mutually perpendicular edges of each video frame are used as two-dimensional plane coordinates, and the coordinates are linearly normalized to obtain the coordinates of each corner point in the normalized coordinate system. The larger the first distance, the greater the positional difference of the matched pairs, the greater the degree of motion, and the smaller the stability feature. The average value of the comprehensive difference degree of the matched pairs and the first distance is calculated to obtain the learning rate adjustment parameters of the matched pairs. The calculation process of the comprehensive difference degree of the matched pairs is the same as that in step S2, except that the pixels at the same position are replaced with the matched pairs. The specific steps are not repeated here. The larger the comprehensive difference degree, the more obvious the comprehensive difference of the matched pairs, and the smaller the stability feature. Furthermore, the larger the learning rate adjustment parameter, the greater the learning rate required for the corner points of the matching pair in the current video frame.

[0047] After obtaining the learning rate adjustment parameters for the corner points with obvious features in the second pixel, for the unmatched points with weak features, it is necessary to determine the pixel points in the previous video frame that may match them, and then analyze the motion degree and learning rate adjustment parameters of the unmatched points. Therefore, the suspected matching points are obtained based on the relative coordinates of the unmatched points and different matching pairs. Preferably, in this embodiment of the invention, the step of obtaining the suspected matching points includes: taking the corner points in the matching pair as the origin, constructing a coordinate system based on the gradient direction of the corner points in the matching pair and the perpendicular line of the gradient direction, with the perpendicular line of the gradient direction pointing to the side with the largest adjacent gray value on both sides of the gradient direction; each matching pair forms a set of coordinate systems in two adjacent video frames. The relative coordinates of the unmatched point in the coordinate system of any corner point in the current video frame are obtained. These relative coordinates are based on the position of the origin of the coordinate system. Since the matching pair of corner points is a pair of relatively similar pixel points, the relative coordinates based on the origin of the coordinate system in the previous video frame may also be the matching pixel points of the unmatched point. Therefore, the pixel points at the relative coordinates in the coordinate system of the previous video frame of the matching pair are taken as the suspected matching points of the unmatched point. Ultimately, each unmatched point can obtain different suspected pairing points based on the coordinate positions of different matching pairs. Thus, the motion degree and learning rate adjustment parameters of the unmatched point can be analyzed among all suspected matching points. Therefore, the learning rate adjustment parameters of the unmatched point are obtained based on the comprehensive difference between the unmatched point and the suspected pairing point and the coordinate distance features.

[0048] Preferably, in this embodiment of the invention, the step of obtaining the learning rate adjustment parameter for unmatched points includes: calculating the minimum value of the overall difference between the unmatched point and all suspected matching points to obtain a first value; if the minimum value of the overall difference is also large, it means that the unmatched point has a greater degree of motion and less stability, and therefore requires a larger learning rate. The linear distance between the coordinates of the suspected matching point and the unmatched point corresponding to the first value is calculated and normalized to obtain a second distance; the larger the second distance, the greater the relative displacement, meaning a greater degree of motion. The average value of the first value and the second distance is calculated to obtain the learning rate adjustment parameter for the unmatched point; the greater the degree of motion of the unmatched point, the larger the learning rate adjustment parameter, and the greater the required learning rate.

[0049] Step S4: Adjust the preset learning rate according to the learning rate adjustment parameters of different pixels in the current video frame to obtain the adaptive learning rate of different pixels; separate the foreground and background of the current video frame according to the MOG2 algorithm and the adaptive learning rate; monitor the smoke according to the separation results.

[0050] After obtaining the learning rate adjustment parameters for corner points and unmatched points in the second pixel, the preset learning rate can be adjusted according to the learning rate adjustment parameters of different pixels in the current video frame to obtain the adaptive learning rate for different pixels. Preferably, in this embodiment of the invention, the step of obtaining the adaptive learning rate includes: calculating the difference between the preset maximum learning rate and the preset minimum learning rate to obtain the adjustment benchmark; calculating the product of the adjustment benchmark and the learning rate adjustment parameter to obtain the adjustment amount; and calculating the sum of the preset minimum learning rate and the adjustment amount to obtain the adaptive learning rate for different pixels in the current video frame. The larger the learning rate adjustment parameter, the larger the adaptive learning rate. The learning rate adjustment parameter for the first pixel is a constant 0. Since the first pixel represents the area in the video frame that has not changed, the learning rate for the first pixel is the preset minimum learning rate. In common application scenarios, the minimum learning rate is generally between 0.001 and 0.005, and the maximum learning rate is generally between 0.05 and 0.1. In this embodiment of the invention, the preset minimum learning rate is 0.001, and the preset maximum learning rate is 0.1. The implementer can determine the value according to the implementation scenario. This yields the adaptive learning rate for all pixels in the current video frame. The adaptive learning rate enables the MOG2 algorithm to more accurately separate the foreground and background.

[0051] Furthermore, the foreground and background of the current video frame can be separated using the MOG2 algorithm and an adaptive learning rate. This adaptive learning rate is embedded in the algorithm to dynamically adjust the model's update parameters, allowing pixels with significant changes to update quickly while stable regions update slowly. This results in more accurate foreground and background separation. It should be noted that the MOG2 algorithm is existing technology, and its specific steps will not be elaborated here. Based on the separation results, smoke is monitored. The separated foreground region and real-time detection data from concentration sensors deployed in the factory are input into a deep learning monitoring network. The network can employ existing spatiotemporal fusion models, such as CNN-LSTM or 3D-CNN structures, to jointly analyze the smoke morphology, diffusion trend, and sensor concentration changes in the video foreground, thereby achieving accurate monitoring and early warning of smoke anomalies. It should be noted that this type of deep learning monitoring network is existing technology, and its specific steps will not be elaborated here. Implementers can determine the fusion monitoring method of video foreground features and sensor data according to the implementation scenario; no limitations are imposed here.

[0052] In summary, this invention provides a deep learning-based method for early detection and warning of factory smoke. It obtains a comprehensive difference degree based on color difference features and neighborhood difference features at the same location in a video frame; obtains a first pixel and a second pixel based on the comprehensive difference degree; performs corner matching between the second pixel and pixels at the same location in the previous video frame to obtain matching pairs and unmatched points; obtains a learning rate adjustment parameter based on the comprehensive difference degree and coordinate distance features of the matching pairs; and obtains a learning rate adjustment parameter for the unmatched points based on the comprehensive difference degree and coordinate distance features between the unmatched points and suspected matching points. This invention adjusts the preset learning rate based on the pixel learning rate adjustment parameter, and uses the MOG2 algorithm and adaptive learning rate to separate the foreground and background of the current video frame and monitor smoke, thus improving the accuracy of smoke detection and warning.

[0053] The present invention also proposes a deep learning-based early monitoring and warning system for factory smoke, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement any of the steps of a deep learning-based early monitoring and warning method for factory smoke.

[0054] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0055] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A deep learning-based method for early monitoring and warning of factory smoke, characterized in that, The method includes the following steps: Acquire video frames from factory monitoring; A color difference feature value is obtained based on the color difference features at the same position between the current video frame and the previous video frame; a comprehensive difference degree is obtained based on the neighborhood difference features at the same position between the current video frame and the previous video frame and the color difference feature value; and a first pixel and a second pixel in the current video frame are obtained based on the comprehensive difference degree. Corner point matching is performed between the second pixel and pixels at the same position in the previous video frame to obtain different matching pairs and unmatched points in the second pixel; the learning rate adjustment parameters of the matching pairs are obtained based on the comprehensive difference degree and coordinate distance features of the matching pairs; suspected matching points are obtained based on the relative coordinates of the unmatched points and different matching pairs; the learning rate adjustment parameters of the unmatched points are obtained based on the comprehensive difference degree and coordinate distance features between the unmatched points and the suspected matching points. The preset learning rate is adjusted according to the learning rate adjustment parameters of different pixels in the current video frame to obtain the adaptive learning rate of different pixels; the foreground and background of the current video frame are separated according to the MOG2 algorithm and the adaptive learning rate; and smoke is monitored according to the separation results.

2. The method for early monitoring and warning of factory smoke based on deep learning according to claim 1, characterized in that, The step of obtaining color difference feature values ​​based on color difference features at the same position between the current video frame and the previous video frame includes: In the formula, D represents the color difference feature value at the same position. This represents the A channel value of the LAB color model for pixels at the same position in the current video frame. This represents the A channel value of the LAB color model for pixels at the same position in the previous video frame. This represents the B channel value of the LAB color model for pixels at the same position in the current video frame. This represents the B channel value of the LAB color model for pixels at the same position in the previous video frame.

3. The method for early monitoring and warning of factory smoke based on deep learning according to claim 1, characterized in that, The step of obtaining the comprehensive difference degree based on the neighborhood difference features of the same position between the current video frame and the previous video frame and the color difference feature value includes: In the formula, W represents the overall degree of difference at the same position. This represents the cosine similarity of feature description vectors at the same location obtained through the SIFT algorithm, where D represents the color difference feature value. This indicates linear normalization.

4. The method for early monitoring and warning of factory smoke based on deep learning according to claim 1, characterized in that, The step of obtaining the first pixel and the second pixel in the current video frame based on the comprehensive difference degree includes: The pixel in the current video frame whose overall difference is a constant of 0 is selected as the first pixel, and the other pixel is selected as the second pixel.

5. The method for early monitoring and warning of factory smoke based on deep learning according to claim 1, characterized in that, The step of performing corner matching between the second pixel and pixels at the same position in the previous video frame to obtain different matching pairs and unmatched points in the second pixel includes: The corner points in the second pixel at the same position as those in the previous video frame are obtained using the FATS corner detection algorithm; the feature description vectors of the corner points are obtained using the SIFT algorithm; the cosine similarity between the feature description vectors of the corner points in the current video frame and any corner point in the previous video frame is calculated; the corner point relationship corresponding to the maximum cosine similarity of each corner point in the current video frame is taken as the matching pair; and the non-corner points in the second pixel are taken as the unmatched points.

6. The method for early monitoring and warning of factory smoke based on deep learning according to claim 1, characterized in that, The step of obtaining the learning rate adjustment parameter of the matching pair based on the comprehensive difference degree and coordinate distance feature of the matching pair includes: Calculate and normalize the straight-line distance between the coordinates of the matching pair to obtain the first distance; calculate the average of the comprehensive difference degree of the matching pair and the first distance to obtain the learning rate adjustment parameter of the matching pair.

7. The method for early monitoring and warning of factory smoke based on deep learning according to claim 1, characterized in that, The step of obtaining suspected matching points based on the relative coordinates of the unmatched points and different matching pairs includes: Using the corner points of the matched pair as the origin, a coordinate system is constructed based on the gradient direction of the corner points and the perpendicular line of the gradient direction. The perpendicular line of the gradient direction points to the side with the largest adjacent gray value on both sides of the gradient direction. The relative coordinates of the unmatched point in the coordinate system of any corner point in the current video frame are obtained, and the pixel point at the relative coordinate in the coordinate system of the previous video frame of the matched pair is taken as the suspected matching point of the unmatched point.

8. The method for early monitoring and warning of factory smoke based on deep learning according to claim 1, characterized in that, The step of obtaining the learning rate adjustment parameter of the unmatched point based on the comprehensive difference degree and coordinate distance feature between the unmatched point and the suspected matching point includes: Calculate the minimum overall difference between the unmatched point and all suspected matching points to obtain a first value; calculate and normalize the straight-line distance between the suspected matching point and the unmatched point corresponding to the first value to obtain a second distance; calculate the average of the first value and the second distance to obtain the learning rate adjustment parameter of the unmatched point.

9. The method for early monitoring and warning of factory smoke based on deep learning according to claim 1, characterized in that, The step of adjusting the preset learning rate according to the learning rate adjustment parameters of different pixels in the current video frame to obtain the adaptive learning rate for different pixels includes: Calculate the difference between the preset maximum learning rate and the preset minimum learning rate to obtain the adjustment benchmark; calculate the product of the adjustment benchmark and the learning rate adjustment parameter to obtain the adjustment amount; calculate the sum of the preset minimum learning rate and the adjustment amount to obtain the adaptive learning rate of different pixels in the current video frame; the learning rate adjustment parameter of the first pixel is a constant 0.

10. A deep learning-based early warning system for factory smoke monitoring, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method as described in any one of claims 1-9.

Images