A smoke detection method based on target detection and disordered features

Abstract

The present invention relates to a smoke detection method based on target detection and disorder characteristics, comprising the following steps: obtaining a current frame image in a surveillance video within an area to be detected; performing smoke detection on the obtained current frame image, first performing image distortion correction, then using a smoke target detection neural network model to extract candidate areas in the image where smoke may exist, then using a smoke semantic segmentation neural network model to perform pixel segmentation on the candidate areas, and calculating a disorder characteristic index of smoke within the candidate areas, determining whether smoke exists in the candidate areas based on the disorder characteristic index, and saving the detection results; if the smoke detection result indicates smoke exists, an alarm is issued. The present invention combines a detection method based on a convolutional neural network with smoke disorder characteristics, achieving effective smoke detection in complex factory environments, solving problems such as poor anti-interference ability, incomplete smoke feature extraction, and low accuracy of existing detection technologies, and improving detection accuracy and stability.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and in particular relates to a smoke detection method based on target detection and disordered features. Background Art

[0002] Chemical plants and other industrial sites often present fire safety hazards due to their large geographical area and small number of workers. Therefore, applying smoke detection in factory settings is of great significance. Smoke detection methods are mainly based on smoke sensors, such as ionization smoke sensors. The sensor contains a radioactive source, americium-241, in the inner and outer ionization chambers. The positive and negative ions generated by ionization migrate toward the positive and negative electrodes respectively under the influence of the electric field. Under normal circumstances, the current and voltage of the inner and outer ionization chambers are stable. If smoke escapes, it will interfere with the normal movement of the charged particles, causing changes in the current and voltage, disrupting the balance between the inner and outer ionization chambers, thereby enabling smoke detection.

[0003] Currently, indoor smoke detection mostly relies on smoke sensors. However, outdoor smoke detection is difficult due to the large spatial range and uncertain smoke movement. Therefore, outdoor scenes rely heavily on manual detection. However, due to the significant human uncertainty and the large number of areas to be inspected, timely detection is difficult to ensure. Intelligent smoke detection equipment often only extracts static smoke features (color, shape, texture, etc.) as a basis for judgment. However, the irregular movement and transparency of smoke make its static features less capable of being detected. As a result, intelligent smoke detection equipment suffers from poor anti-interference capabilities, incomplete smoke feature extraction, and low accuracy. Summary of the Invention

[0004] The purpose of the present invention is to provide a smoke detection method based on target detection and disorder characteristics, which is used to achieve effective detection of smoke in complex environments.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A smoke detection method based on target detection and disorder features includes the following steps:

[0007] S1: Obtain the current frame image in the surveillance video within the area to be detected;

[0008] S2: Perform smoke detection on the current frame image, including:

[0009] (1) Image preprocessing, distortion correction of the obtained image,

[0010] (2) Using the smoke target detection neural network model to extract candidate areas where smoke may exist in the image,

[0011] (3) Use the smoke semantic segmentation neural network model to segment the candidate area into pixels and calculate the disorder characteristic index of the smoke in the candidate area.

[0012] (4) Determine whether there is smoke in the candidate area based on the disorder characteristic index,

[0013] (5) Save the test results;

[0014] S3: If the smoke detection result shows that there is smoke, an alarm is issued, otherwise the process returns to step S1 to continue detection.

[0015] Preferably, in S2(1), the radial distortion correction formula used for distortion correction is:

[0016] x0=x(1+k1r 2 +k2r 4 +k3r 6 ),

[0017] y0=y(1+k1r 2 +k2r 4 +k a r 6 ),

[0018] in:

[0019] (x0, y0) is the original position of the distorted pixel,

[0020] (x, y) is the position after distortion correction,

[0021] k1, k2, k3 are distortion coefficients,

[0022] r=x 2 +y 2 .

[0023] Preferably, in S2(2), the smoke target detection neural network model includes a Faster R-CNN network, or an R-FCN network, or a YOLO network.

[0024] Preferably, in S2(2), the smoke target detection neural network model is obtained by the following steps:

[0025] (1) Obtain images of surveillance videos with smoke as training samples;

[0026] (2) Manually label the smoke area in the image, including location and type;

[0027] (3) Randomly shuffle the labeled images and divide them into training set, validation set and test set according to the set ratio;

[0028] (4) The labeled information and images are input into the neural network model for training. After completing the set conditions, a smoke target detection neural network model is obtained.

[0029] Further preferably, there are multiple set conditions, and after completing at least one set condition, a smoke target detection neural network model is obtained.

[0030] Further preferably, the setting conditions include:

[0031] (1) The actual training time exceeds the preset training time;

[0032] (2) The actual number of training sessions exceeds the preset number of training sessions;

[0033] (3) The difference calculated by the loss function is less than the preset difference threshold.

[0034] Preferably, in S2(3), the step of performing pixel segmentation on the candidate area using the smoke semantic segmentation neural network model is as follows:

[0035] (1) Extract the candidate region from the image;

[0036] (2) Inputting the intercepted candidate region image into the smoke semantic segmentation neural network model;

[0037] (3) According to the smoke semantic segmentation neural network model, the number of smoke pixels in the candidate area is obtained and the result is output.

[0038] Further preferably, the smoke semantic segmentation neural network model determines whether it is a smoke pixel based on the color and spatial distribution characteristics of a single pixel in the candidate area, and obtains the number of smoke pixels.

[0039] Preferably, in S2(3), the disorder characteristic index is:

[0040]

[0041] in:

[0042] c i is the characteristic index of disorder,

[0043] N s is the number of smoke pixels,

[0044] N all is the total number of pixels in the candidate area,

[0045] w i 、h i are the width and height of the candidate region.

[0046] Preferably, in S2(4), the method for judging whether there is smoke in the candidate area based on the disorder characteristic index includes: comparing the disorder characteristic index of the current frame image with the disorder characteristic index of the previous frame image; if the fluctuation is greater than the set ratio, then there is smoke in the candidate area of the current frame image; otherwise, there is no smoke.

[0047] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0048] The present invention combines the detection method based on convolutional neural networks and the disordered characteristics of smoke, realizing effective smoke detection in complex factory environments. It solves the problems of poor anti-interference ability, incomplete smoke feature extraction, and low accuracy in existing smoke detection technologies, improves the accuracy and stability of detection, and the process is simple and easy to implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Attachment Figure 1 is an overall flow chart of the detection method of this embodiment;

[0050] Attachment Figure 2 Flowchart of the detection process of this embodiment;

[0051] Attachment Figure 3 is a schematic diagram of the smoke candidate area of this embodiment;

[0052] Attachment Figure 4 This is a schematic diagram of a candidate region captured from an image in this embodiment;

[0053] Attachment Figure 5 Schematic diagram of smoke pixel distribution in the candidate area of this embodiment. DETAILED DESCRIPTION

[0054] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0056] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0057] A smoke detection method based on target detection and disordered features, such as Figure 1 As shown, the following steps are included:

[0058] S1: Obtain the current frame image in the surveillance video in the area to be detected. Specifically: obtain the IP address of the camera in the factory area where smoke detection is required, obtain the surveillance video under the IP address, and decode and extract the surveillance video to obtain the current frame image to be detected.

[0059] S2: Perform smoke detection on the current frame image obtained, such as Figure 2 Shown, including:

[0060] (1) Image preprocessing: distortion correction of the obtained image. Specifically:

[0061] The images in the surveillance video captured by the camera have certain distortion, especially at the edge of the line of sight, where the distortion is more serious. The imaging process of the camera is essentially a coordinate system conversion. First, the point in space is converted from the "world coordinate system" to the "camera coordinate system", and then it is projected onto the imaging plane (image physical coordinate system). Finally, the data on the imaging plane is converted to image pixel coordinates. However, due to the deviation of the lens manufacturing precision and the assembly process, distortion will be introduced, resulting in the distortion of the original image. Distortion can be divided into radial distortion and tangential distortion. In surveillance videos, radial distortion is usually the case. Therefore, a radial distortion correction formula is used to correct the distortion to improve the accuracy of detection and prevent the distortion from affecting the pixel segmentation in the subsequent detection steps, making the distribution of smoke pixels inconsistent with the actual situation.

[0062] The radial distortion correction formula is as follows:

[0063] x0=x(1+k1r 2 +k2r 4 +k3r 6 ),

[0064] y0=y(1+k1r 2 +k2r 4 +k3r 6 ),

[0065] in:

[0066] (x0, y0) is the original position of the distorted pixel,

[0067] (x, y) is the position after distortion correction,

[0068] k1, k2, k3 are distortion coefficients,

[0069] r=x 2 +y 2 .

[0070] (2) A smoke target detection neural network model is used to extract candidate areas in the image where smoke may exist. Specifically:

[0071] The specific steps of using the smoke target detection neural network model to extract candidate areas where smoke may exist in an image include:

[0072] 1. Input the preprocessed image into the smoke target detection neural network model,

[0073] 2. The output of the smoke target detection neural network model is: if there is smoke, the location of the smoke is framed in the image with a rectangular box, which is the candidate area of the smoke. Figure 3 As shown;

[0074] The smoke target detection neural network model can use the Faster R-CNN network, R-FCN network, or YOLO network. The smoke target detection neural network model is obtained through the following steps:

[0075] 1. Obtain images from surveillance videos with smoke as training samples. To ensure consistency between neural network training and detection results, the training samples are corrected for radial distortion in the same way as in the above steps.

[0076] 2. Manually mark the smoke area in the image, including the location and type of smoke.

[0077] 3. Randomly shuffle the labeled images and divide them into training set, validation set and test set according to the set ratio. The set ratio can be 6:2:2.

[0078] 4. The labeled information and image are input into the neural network model for training. The neural network obtains the location of the smoke as the network output through forward propagation calculation and compares it with the labeled information. The parameters are updated through backpropagation according to the correct location of the smoke and the current predicted location of the network, so that the extracted image features are closer and closer to the smoke features. After completing at least one of the multiple set conditions, a smoke target detection neural network model is obtained. The set conditions include: the actual training time exceeds the preset training time, the actual training number exceeds the preset training number, and the difference calculated by the loss function is less than the preset difference threshold.

[0079] Of course, in some embodiments, it is also possible not to extract candidate areas where smoke may exist in the image, but to directly perform pixel segmentation on the entire image using the smoke semantic segmentation neural network model. However, by using the smoke target detection neural network model to extract candidate areas where smoke may exist in the image, the processing scope of subsequent pixel segmentation of the candidate areas using the smoke semantic segmentation neural network model can be narrowed, the amount of calculation is reduced, the calculation rate is improved, and it is more convenient to calculate the disorder feature index.

[0080] (3) The smoke semantic segmentation neural network model is used to segment the candidate area into pixels and calculate the disorder characteristic index of the smoke in the candidate area. Specifically:

[0081] The neural network models used include Mask-RCNN and others. The method for obtaining the smoke semantic segmentation neural network model is similar to that of the smoke target detection neural network model, so it will not be repeated here.

[0082] The specific steps of pixel segmentation of candidate areas using the smoke semantic segmentation neural network model include:

[0083] 1. Cut out the candidate area from the image, such as Figure 4 As shown,

[0084] 2. Input the intercepted candidate area image into the smoke semantic segmentation neural network model.

[0085] 3. The smoke semantic segmentation neural network model determines whether it is a smoke pixel based on the color and spatial distribution characteristics of a single pixel in the candidate area, such as Figure 5 As shown, the number of smoke pixels is obtained;

[0086] The calculation formula of the disorder characteristic index is:

[0087]

[0088] in:

[0089] c i is the characteristic index of disorder,

[0090] N s is the number of smoke pixels,

[0091] N all is the total number of pixels in the candidate area,

[0092] w i 、h i are the width and height of the candidate region.

[0093] (4) Determine whether there is smoke in the candidate area based on the disorder characteristic index. However, it is impossible to determine whether there is disorder based on the disorder index of the current frame image alone. It is necessary to compare the disorder index of the previous frame image at the same time to make the smoke detection effect more stable. Specifically: compare the disorder characteristic index of the current frame image with the disorder characteristic index of the previous frame image. If the fluctuation is greater than the set ratio, there is smoke in the candidate area of the current frame image, otherwise there is no smoke. The set ratio is usually 5%.

[0094] (5) Saving the detection results, which include the original surveillance video image of the location where the smoke has been marked, the location information of the smoke in the image, and the confidence level.

[0095] S3: If the smoke detection result shows that there is smoke, an alarm is sounded to remind the staff to check, otherwise the process returns to step S1 to continue the detection. After the staff turns off the alarm, the process returns to step S1 to continue the detection.

[0096] Through experimental tests using the above-mentioned smoke detection method, it can be seen that: when the smoke target detection neural network model is used to roughly locate the candidate areas where smoke may exist in the image, the missed detection rate of real smoke is 0%; when the smoke semantic segmentation neural network model performs pixel segmentation on the candidate areas, the pixel classification accuracy is 99.5%, that is, the accuracy of the number of smoke pixels is 99.5%; when judging based on the disordered characteristic indicators of smoke, the accuracy rate is increased by nearly 45% compared with the direct use of the target detection method, that is, the accuracy rate is 99.5%, which improves the accuracy and stability of detection; at the same time, the time for detecting a frame of video image using this method is less than 0.5 seconds, the detection speed is extremely fast, and the purpose of effective smoke detection in complex factory environments is achieved.

[0097] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A smoke detection method based on target detection and disorder characteristics, characterized by: The steps include: S1: Obtain the current frame image in the surveillance video within the area to be detected; S2: Perform smoke detection on the current frame image, including: (1) Image preprocessing, distortion correction of the obtained image, (2) Using the smoke target detection neural network model to extract candidate areas where smoke may exist in the image, (3) Use the smoke semantic segmentation neural network model to perform pixel segmentation on the candidate area. The steps are as follows: Extract the candidate region from the image; The captured candidate area image is input into the smoke semantic segmentation neural network model; The smoke semantic segmentation neural network model determines whether it is a smoke pixel based on the color and spatial distribution characteristics of a single pixel in the candidate area, and obtains the number of smoke pixels. The smoke semantic segmentation neural network model obtains the number of smoke pixels in the candidate area and outputs the result. Calculate the disorder characteristic index of the smoke in the candidate area. The disorder characteristic index is: in: c i is the disorder characteristic indicator, N s is the number of smoke pixels, N all is the total number of pixels in the candidate area, w i 、h i are the width and height of the candidate region, (4) judging whether there is smoke in the candidate area according to the disorder characteristic index, including: comparing the disorder characteristic index of the current frame image with the disorder characteristic index of the previous frame image, if the fluctuation is greater than a set ratio, then there is smoke in the candidate area of the current frame image, otherwise there is no smoke; (5) Save the test results; S3: If the smoke detection result shows that there is smoke, an alarm is issued, otherwise the process returns to step S1 to continue detection.

2. The smoke detection method based on target detection and disorder characteristics according to claim 1, characterized in that: In S2(1), the radial distortion correction formula used for distortion correction is: x0=x(1+k1r 2 +k2r 4 +k3r 6 ), y0=y(1+k1r 2 +k2r 4 +k3r 6 ), in: (x0, y0) is the original position of the distorted pixel, (x, y) is the position after distortion correction, k1, k2, k3 are distortion coefficients, r=x 2 +y 2 。 3. The smoke detection method based on target detection and disorder characteristics according to claim 1, characterized in that: In S2(2), the smoke target detection neural network model includes a Faster R-CNN network, or an R-FCN network, or a YOLO network.

4. The smoke detection method based on target detection and disorder characteristics according to claim 1, characterized in that: In S2(2), the smoke target detection neural network model is obtained through the following steps: (1) Obtain images of surveillance videos with smoke as training samples; (2) Manually label the smoke area in the image, including location and type; (3) Randomly shuffle the labeled images and divide them into training set, validation set and test set according to the set ratio; (4) The labeled information and images are input into the neural network model for training. After completing the set conditions, a smoke target detection neural network model is obtained.

5. The smoke detection method based on target detection and disorder characteristics according to claim 4, characterized in that: There are multiple setting conditions, and after completing at least one setting condition, a smoke target detection neural network model is obtained.

6. The smoke detection method based on target detection and disorder characteristics according to claim 5, characterized in that: The setting conditions include: (1) The actual training time exceeds the preset training time; (2) The actual number of training sessions exceeds the preset number of training sessions; (3) The difference calculated by the loss function is less than the preset difference threshold.

Images