Image-based smoke detection system and method
By constructing complex neural network trees and image processing technology, the problem of high false alarm rate of photoelectric smoke detectors is solved, and the effective elimination of lens accumulation, fog and vibration is achieved, which significantly improves the accuracy and reliability of smoke detection. .
Patent Information
- Application Number
- CN202010832357.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-08-18
AI Technical Summary
Existing photoelectric smoke detectors are susceptible to interference from air impurities, feathers, dust, etc., resulting in a high false alarm rate and failing to effectively eliminate false alarms caused by factors such as lens accumulation, fog, and vibration.
Multiple groups of simple binary neural networks are used to form a complex neural network tree through logical operations, combining the camera unit, image preprocessing unit, smoke static feature recognition module, smoke dynamic feature recognition module, image storage unit and control unit, and through logical operations and images Processing technology to reduce the probability of false alarms.
It significantly reduces the probability of false alarms caused by lens dust accumulation, fog and vibration, and improves the accuracy and reliability of smoke detection.
Smart Images

Figure CN112232107B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and more specifically, to a smoke detection system and method based on image recognition. Background Art
[0002] The detection system is used to monitor whether there is smoke that indicates a fire, so that the monitor can take timely measures to prevent the fire. Today, the photoelectric smoke detectors used are made based on the principle of smoke refraction of light, which is easily interfered by air impurities, feathers and dust. Statistics show that the false alarm rate is as high as 90%. Therefore, in the process of developing the smoke detection system, in order to solve the problem of too high false alarm rate of the detection system, an image-based smoke detection system and method are proposed in the art, as follows:
[0003] Patent US7542585 disclosed in November 2005 a fire detection method by identifying pixels of fire and smoke in an image.
[0004] Patent CN101794450 disclosed in November 2009 a method for determining whether a point is a smoke point based on the optical flow direction of each pixel in a differential image between sequential frames of a video image.
[0005] Patent CN102136059 disclosed a video-based smoke detection method in March 2011, which integrates multiple methods such as background modeling to extract moving objects, support vector machine (SVM) classifier to screen gray and white areas, wavelet transform high-frequency signal change analysis, and smoke texture characteristic cascade classifier to achieve smoke detection.
[0006] Patent CN108985192 disclosed in June 2018 a video smoke recognition method based on a multi-task deep convolutional neural network. It adopts a multi-task deep learning architecture consisting of a convolutional layer, a fully connected layer, and a deconvolutional layer. Through a large number of training samples, it learns the static information of a single frame image in the video and the dynamic optical flow information between frames, and then determines whether the test video frame contains smoke.
[0007] Patent CN109977790 disclosed a video smoke detection and recognition method based on transfer learning in March 2019. The improved VGG-16 network was pre-trained using the ImageNet image dataset, allowing the network to learn the ability to extract deep features from natural images and obtain transfer features; the Faster R-CNN network was trained on the annotated smoke dataset using transfer learning to identify smoke.
[0008] However, the above method and system do not include steps and units for eliminating false alarms caused by factors such as lens dust accumulation, fog, vibration, etc., resulting in a high false alarm rate caused by factors such as lens dust accumulation, fog, vibration, etc. In view of the above defects in the field, the present invention proposes an improved smoke detection method and system, which is composed of multiple groups of simple binary classification neural networks through logical operations to form a complex neural network tree, which can significantly reduce the probability of false alarms caused by lens dust accumulation, fog and vibration. Summary of the invention
[0009] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
[0010] Traditional photoelectric smoke detectors are contact point detectors, while image detectors are non-contact space detectors. Micro camera components, combined with batteries, can be easily placed at various locations that need to be monitored, such as cargo hold wall panels. Multiple groups of components form a space monitoring network that can capture smoke at the ignition stage without the need for smoke to diffuse to the detector location, which improves the detection speed. Micro camera components are very mature in industrial applications and are extremely cheap, which has a great cost advantage over photoelectric ones.
[0011] In one embodiment of the present invention, a system for image-based smoke detection is provided, comprising:
[0012] A camera unit for capturing images;
[0013] An image pre-processing unit, which is used to process the captured image into a binary image;
[0014] Image contrast unit, comprising:
[0015] a smoke static feature recognition module, comprising a four-classification neural network for classifying the captured image and a corresponding two-classification neural network for classifying the classified image into smoke or no smoke, wherein the four-classification neural network classifies the captured image into a normal lens image, a dusty lens image, a fogged lens image, and a vibrated lens image; and
[0016] A smoke dynamic feature recognition module, which is used to determine whether there is smoke in the captured image by subtracting the binary image from the background image to extract edge features and accumulating the extracted edge features;
[0017] an image storage unit, configured to store the captured image; and
[0018] A control unit controls the above units to perform corresponding operations, and generates a smoke alarm signal when both the smoke static feature recognition module and the smoke dynamic feature recognition module determine that there is smoke.
[0019] In one embodiment of the present invention, the system further includes a wireless transmission unit, and the camera unit transmits the captured image to the image pre-processing unit via the wireless transmission unit, and the camera unit and the wireless transmission unit together constitute a miniature camera assembly arranged in the smoke flow channel.
[0020] In one embodiment of the present invention, the wireless transmission unit uses at least one of Bluetooth, Wi-Fi, 3G, 4G and 5G or other transmission methods in the art. Preferably, the use of 5G network can transmit a larger amount of data and support more component networking.
[0021] In one embodiment of the present invention, the image pre-processing unit is further used to perform denoising and gray-scaling on the captured image. Denoising helps to more accurately identify the image, while gray-scaling can be used to determine whether the captured image is clear enough.
[0022] The scene captured by the camera unit may vibrate due to various reasons. As an example and not limitation, cargo is transported in a fixed position in the cargo hold of an aircraft. When it is slightly loose or the aircraft vibrates, the captured image will "shake" a little. In this case, if a single picture is subtracted from the background image and edge features are extracted to perform smoke judgment, it will be very easy to make a misjudgment. Therefore, in one embodiment of the present invention, accumulating the extracted edge features includes subtracting a plurality of consecutive captured images from the background image to determine whether one or more of the luminous flux change rate, the smoke area change rate, and the smoke edge length change rate is greater than a set second threshold.
[0023] In one embodiment of the present invention, the four-classification neural network is also used to give the possibility that the captured image is a normal lens image, a dusty lens image, a fogged lens image, or a vibrating lens image, so that the captured image enters the corresponding two-classification neural network with the maximum possibility to obtain the possibility of smoke, and the smoke static feature recognition module determines that there is smoke in the captured image when the product of the maximum possibility and the smoke possibility is greater than the set first threshold. Therefore, the present invention improves the accuracy of judgment and reduces the probability of false alarms through a lightweight simple two-classification neural network through a combination of different logics.
[0024] In one embodiment of the present invention, the image contrast unit also includes a background image update module, which updates the background image by the following steps: setting an image counter; resetting the image counter when the image counter accumulates to a threshold number of images; selecting an image with the lowest possibility of smoke from the threshold number of images, wherein the possibility of smoke in the image comes from the possibility of smoke obtained via the binary classification neural network; and obtaining a binary image of the selected image via the image pre-processing unit and updating it as the background image.
[0025] In one embodiment of the present invention, the system further comprises a lighting unit, which adopts a cold light source and switches the light color based on whether the captured image is clear.
[0026] In one embodiment of the present invention, the image pre-processing unit includes an image fusion module, which performs the following steps: subtracting the binary image of the captured image from the background image to obtain a difference image; using a grayscale change function to calculate the grayscale change rate of the difference image as a whole; using a gradient function to calculate the edge of the smoky part of the captured image and determine the edge gradient change rate; determining that the captured image is unclear when the grayscale change rate is greater than a set third threshold and the edge gradient change rate is greater than a set fourth threshold; instructing the lighting unit to switch the light color according to red, yellow and blue via the control unit to obtain three images of different colors; and weighted summing the three images of different colors according to structural similarity to obtain a fused image for image comparison.
[0027] In one embodiment of the present invention, the system has a standby mode and a monitoring mode, in which the camera unit captures images at a low frequency in the standby mode, and when the first smoke image is identified, the system enters a monitoring mode and sets a monitoring cycle, in which the camera unit captures images at a high frequency.
[0028] In one embodiment of the present invention, the image storage unit classifies and saves the captured images into four categories: normal lens image, dusty lens image, fogged lens image and lens vibration image, and uploads them to a cloud platform for the training modules for the four-category neural network and the two-category neural network in the cloud platform to perform training and deep learning based on the uploaded images, and the trained and learned four-category neural network and two-category neural network are downloaded to the image comparison unit.
[0029] In another embodiment of the present invention, a method for image-based smoke detection is provided, comprising:
[0030] Capture images;
[0031] Processing the captured image into a binary image;
[0032] Determining whether there is smoke in the captured image by subtracting the binary image from a background image to extract edge features and accumulating the extracted edge features;
[0033] Performing four-classification on the captured image and classifying the image after the four-classification into smoke or no smoke, wherein the four-classification classifies the captured image into a normal lens image, a dusty lens image, a fogged lens image, and a vibrated lens image; and
[0034] A smoke alarm signal is generated when it is determined that there is smoke in the captured image through both edge feature accumulation and image classification.
[0035] Aspects of the present disclosure generally include methods and systems as substantially described herein with reference to and as illustrated by the accompanying drawings. As can be appreciated by those skilled in the art, the technical solutions or partial solutions of the present invention can be adaptively adjusted according to the scene, and can be widely applied to the early forecasting of cargo holds, venues, exhibition halls, forest fires, etc.
[0036] After studying the description of the specific exemplary embodiments of the present invention below in conjunction with the accompanying drawings, other aspects, features and embodiments of the present invention will be apparent to those of ordinary skill in the art. Although features of the present invention may be discussed below with respect to certain embodiments and accompanying drawings, all embodiments of the present invention may include one or more of the advantageous features discussed herein. In other words, although one or more embodiments may be discussed to have certain advantageous features, one or more of such features may also be used according to the various embodiments of the present invention discussed herein. In a similar manner, although exemplary embodiments may be discussed below as device, system or method embodiments, it should be appreciated that such exemplary embodiments may be implemented in various devices, systems, and methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to understand in detail the manner in which the above-stated features of the present disclosure are used, a more particular description of the content briefly summarized above may be made with reference to various aspects, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only certain typical aspects of the present disclosure and are not to be considered limiting of its scope, as the description may admit to other equally effective aspects.
[0038] Figure 1 A block diagram of an image-based smoke detection system according to an embodiment of the present invention is shown;
[0039] Figure 2A processing flow chart of a smoke dynamic feature recognition module in an image comparison unit according to an embodiment of the present invention is shown;
[0040] Figure 3 A processing flow chart of a smoke static feature recognition module in an image comparison unit according to an embodiment of the present invention is shown;
[0041] Figure 4 A processing flow chart of a background image updating module in an image comparison unit according to an embodiment of the present invention is shown;
[0042] Figure 5 A processing flow chart of an image fusion module in an image pre-processing unit according to an embodiment of the present invention is shown; and
[0043] Figure 6 A flow chart of an image-based smoke detection method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0044] The various embodiments will be described in more detail below with reference to the accompanying drawings that form part of the present invention and illustrate various specific exemplary embodiments. However, the various embodiments can be implemented in many different forms and should not be interpreted as limiting the various embodiments set forth herein; on the contrary, these embodiments are provided to make the present disclosure thorough and complete, and to fully convey the scope of these embodiments to those of ordinary skill in the art. The various embodiments may be implemented in accordance with methods, systems, or devices. Therefore, these embodiments may be implemented in hardware, in full software, or in a combination of software and hardware. Therefore, the following specific embodiments are not limiting.
[0045] The steps in each flow chart may be performed by hardware (e.g., processor, engine, memory, circuit), software (e.g., operating system, application, driver, machine / processor executable instructions), or a combination thereof. As will be appreciated by those of ordinary skill in the art, each embodiment may include more or fewer steps than shown.
[0046] Figure 1 The block diagram of an image-based smoke detection system 100 according to an embodiment of the present invention is shown. The image-based smoke detection system 100 comprises a control unit 110, a camera unit 102, an image pre-processing unit 104, an image comparison unit 106, an image storage unit 108, and an optional cloud platform 112.
[0047] The control unit 110 controls the functions or operations of the remaining units via control signals. The control unit 110 may include an instruction register, an instruction decoder, an operation controller, and any other necessary components known in the art. According to the program pre-programmed by the user, each instruction is taken out from the memory in turn and placed in the instruction register. The instruction decoding (analysis) determines what operation should be performed, and then the operation controller sends an operation control signal to the corresponding component according to the determined timing. The operation controller mainly includes control logic such as a beat pulse generator, a control matrix, a clock pulse generator, a reset circuit, and a start-stop circuit. As can be understood by those skilled in the art, the control unit 110 can be implemented in any other manner known in the art, and the protection scope of the present invention is not limited to the above-mentioned implementation.
[0048] The camera unit 102 includes a camera module, which may include multiple micro-camera components. These micro-camera components can be conveniently arranged at various locations where smoke needs to be monitored in conjunction with batteries. In one embodiment of the present invention, as an example but not a limitation, the camera module can be arranged on the wall panel of the aircraft cargo hold, and multiple micro-camera components can form a space monitoring network so that smoke can be captured at the smoke ignition stage without waiting for the smoke to diffuse to the location of the micro-camera components, thereby improving the monitoring efficiency and detection speed of the image-based smoke detection system 100.
[0049] In another embodiment of the present invention, the system 100 may further optionally include a wireless transmission unit (not shown). Figure 1 ), the camera unit 102 can transmit the captured image to the image pre-processing unit 104 through the wireless transmission unit, and the camera unit 102 and the wireless transmission unit together constitute a miniature camera assembly arranged at a desired location (such as in a smoke flow channel). In one embodiment of the present invention, the wireless transmission unit uses at least one of Bluetooth, Wi-Fi, 3G, 4G and 5G. Preferably, the use of a 5G network can transmit a larger amount of data and can support more component networking.
[0050] In one embodiment of the present invention, the image-based smoke detection system 100 has a standby mode and a monitoring mode. As an example and not a limitation, when the aircraft cargo door is closed, the system may enter the standby mode. In the standby mode, the control unit 110 controls the camera unit 102 to capture images at a low frequency (such as once every 5 seconds), and when the first smoke image is identified, the system enters the monitoring mode and sets the monitoring cycle (such as 10 seconds), and in the monitoring mode, the camera unit 102 captures images at a high frequency (such as once every 0.5 seconds). As can be understood by those skilled in the art, the frequency at which the camera unit 102 captures images, including low frequency and high frequency, can be set as needed, and the scope of protection of the present invention is not limited to the above frequency. During the monitoring cycle, if smoke is identified again, the monitoring cycle sequence is extended.
[0051] The camera unit 102 transmits the captured image to the image pre-processing unit 104. The image pre-processing unit 104 can be used to process the captured image into a binary image for image comparison. Figure 2 Detailed description. Processing the captured image into a binary image includes setting the grayscale value of the pixel on the image to 0 or 255. The grayscale value of each pixel in the image can only be 0 or 1, that is, it is either black or white. The binary image can be understood as a black and white image, that is, the entire image presents an obvious visual effect of only black and white. It can be understood by those skilled in the art that other image binarization methods known in the art can also be used to process the captured image into a binary image, and the protection scope of the present invention is not limited to a certain method.
[0052] The image pre-processing unit 104 can also be used to denoise and grayscale the captured image. Denoising helps to more accurately identify the image, while grayscale can be used to determine whether the captured image is clear enough. In one embodiment of the present invention, when the captured image is not clear enough, the image fusion module (optional and not included in the image pre-processing unit 104) can be used. Figure 1 The image fusion process is combined with the illumination unit (not shown) that may be optionally included in the system 100. Figure 1 The lighting unit adopts a cold light source (such as an LED light group) and switches the light color based on whether the captured image is clear. Since different colors of light have different penetration into smoke, fusing images of different colors through image processing can improve image quality and help extract features, which will be combined with Figure 5 Describe in more detail.
[0053] The image contrast unit 106 includes a smoke static feature recognition module and a smoke dynamic feature recognition module. The binary image generated by the image pre-processing unit 104 enters the smoke dynamic feature recognition module, and the original image that has not been processed by the image pre-processing unit 104 directly enters the smoke static feature recognition module.
[0054] The smoke static feature recognition module includes a four-classification neural network for classifying the captured image and a corresponding two-classification neural network for classifying the classified image into smoke and no smoke, and the four-classification neural network classifies the captured image into a normal lens image, a dusty lens image, a fogged lens image, and a vibrating lens image. The smoke dynamic feature recognition module is used to determine whether there is smoke in the captured image by subtracting the binary image from the image pre-processing unit 104 from the background image to extract edge features and accumulating the extracted edge features.
[0055] The image comparison unit 106 then transmits the smoke / no smoke determination results generated by the smoke static feature recognition module and the smoke dynamic feature recognition module to the control unit 110. The control unit 110 generates a smoke alarm signal when both the smoke static feature recognition module and the smoke dynamic feature recognition module determine that there is smoke. In one embodiment of the present invention, in an aircraft cargo hold scenario as an example but not a limitation, the control unit transmits the smoke alarm signal to other aircraft systems through the avionics system. The specific operation procedures of the smoke static feature recognition module and the smoke dynamic feature recognition module will be respectively combined with Figure 2 and 3 Describe in detail.
[0056] In another embodiment of the present invention, the image contrast unit 106 may also optionally include a background image updating module (not shown in the figure for simplicity). Figure 1 The background image update module is used to update the background image used by the smoke dynamic feature recognition module to compare with the captured binary image. The operation flow of the background image update module will be described below in conjunction with Figure 4 Detailed description.
[0057] Each image captured by the camera unit 102 is stored in the image storage unit 108. The image storage unit 108 classifies and saves the captured images according to four categories: normal lens image, dust accumulation lens image, fogged lens image, and lens vibration image (classified by the smoke static feature recognition module in the image comparison unit 106) and uploads them to the cloud platform 112 (together with the classification labels generated by the image comparison unit 106) for the training module 114 for the four-classification neural network and the two-classification neural network in the cloud platform 112 to perform training and deep learning based on the uploaded images, and the trained and learned four-classification neural network and two-classification neural network are downloaded to the image comparison unit 106 for image classification.
[0058] Thus, through the cloud platform 112, the collected images are uploaded to the cloud platform for model training to achieve intelligent upgrades. The more smoke images are collected, the higher the accuracy of smoke recognition will be, and the probability of false alarm will be lower and lower. In the aircraft cargo hold scenario, which is used as an example but not a limitation, when the aircraft lands on the apron, the images collected during the flight are uploaded to the cloud platform 112 for neural network training. In addition, in scenarios such as venues or exhibition halls with good network conditions, the collected images can be regularly uploaded to the cloud platform 112 for neural network training, and the four-category and two-category neural networks in the image comparison unit 106 can also be regularly updated.
[0059] As can be understood by those skilled in the art, Figure 1 The cloud platform 112 and the neural network training module 114 in the cloud platform are optional, and any other methods known in the art can be used to implement the update and iteration of the neural network and should be included in the scope of protection of the present invention.
[0060] Figure 2 The figure shows a processing flow chart of the smoke static feature recognition module 212 in the image comparison unit according to an embodiment of the present invention.
[0061] The smoke static feature recognition module 212 is essentially a combined deep neural network, comprising the following operation steps:
[0062] 1) The i-th picture from the camera unit 102 enters the smoke static feature recognition module 212, specifically, first enters a four-classification neural network 202, which can classify the image captured from the camera unit into a normal lens image, an image captured when the lens is dusty, an image captured when the lens is fogged, and an image captured when the lens is vibrating, and gives the probability of these four images q1, q2, q3, q4 respectively;
[0063] 2) Find max(q1,q2,q3,q4), that is, take the maximum value among q1, q2, q3, q4;
[0064] 3) If q1 is the largest, the i-th picture enters the normal lens with smoke or no smoke binary classification neural network 204, and the probability of smoke x1 is given;
[0065] 4) If q2 is the largest, the i-th picture enters the lens dust accumulation smoke or no smoke binary classification neural network 206, and the probability of smoke x2 is given;
[0066] 5) If q3 is the largest, the i-th picture enters the lens fogging smoke or no smoke binary classification neural network 208, and the probability of smoke is x3;
[0067] 6) If q4 is the largest, the i-th picture enters the lens vibration smoke or no smoke binary classification neural network 210, and the possibility of smoke x4 is given.
[0068] 7) If q1*x1 or q2*x2 or q3*x3 or q4*x4 is greater than the set first threshold, it is determined that there is smoke, otherwise there is no smoke.
[0069] As can be understood by those skilled in the art, the setting of the first threshold value can be set by those skilled in the art according to actual needs, and the protection scope of the present invention is not limited to one or more values. The four-classification neural network 202 and the two-classification neural network 204-210 can use any suitable deep neural network or learning model, and can be combined with any suitable linear classifier and logistic regression algorithm or other methods to derive the image classification probability q1-q4 and the smoke probability x1-x4. Any suitable deep neural network or learning model mentioned above and any suitable linear classifier and logistic regression algorithm or other methods of deriving probabilities should be included in the protection scope of the present invention.
[0070] Therefore, for the four dimensions of normal lens, dust accumulation on the lens, fogging of the lens and vibration of the lens, the four-classification neural network and the lightweight simple two-classification neural network improve the accuracy of smoke judgment and reduce the probability of false alarms through the combination of different logics.
[0071] Figure 3 A processing flow chart of the smoke dynamic feature recognition module 302 in the image comparison unit according to an embodiment of the present invention is shown.
[0072] The binary image generated by the image pre-processing unit 104 is transmitted to the smoke dynamic feature recognition module 302 in the image contrast unit 106 .
[0073] In step 304, the received binary image is subtracted from the background image to obtain a difference image.
[0074] In step 306, edge features are extracted from the differential image. In one embodiment of the present invention, the edge contour of the smoke area is obtained by using a well-known edge detection algorithm, and the edge contour is the boundary of the detection area, and the area surrounded by the edge contour is the detection area. The edge detection algorithm may include any one of the algorithms such as the gradient method, the Roberts gradient method, the Sobel algorithm, the Laplace algorithm, and the Canny algorithm. In another embodiment of the present invention, the grayscale change in a certain neighborhood of each pixel of the image is considered, and the first-order or second-order derivative change law of the edge is used to set a gradient operator for a certain neighborhood of the pixel in the original image. Usually, a small area template is used for convolution to calculate, and there are Sobel operators, Robinson operators, Laplace operators, etc. As those skilled in the art can understand, in addition to the above edge feature extraction method, any other suitable method can be used to extract edge features. The protection scope of the present invention is not limited to the above specific edge feature extraction method, but covers all methods or methods known in the art that can be used for edge feature extraction.
[0075] In step 308, multiple consecutive images can be subtracted from the background image to accumulate edge features, such as calculating the rate of change of light flux, the rate of change of edge length, or the rate of change of smoke area. Since smoke not only has an overall flow direction after it is generated, but also slowly spreads to the surroundings, in one embodiment of the present invention, the presence of smoke can be determined by calculating the rate of change of the edge contour length (number of edge pixels) of the suspected smoke area.
[0076] In the early stage of smoke generation, the area of smoke generally shows a continuous growth characteristic. And as the material burns, the temperature continues to rise, accelerating the movement of smoke molecules, causing the area of smoke to further grow. Therefore, in another embodiment of the present invention, the presence of smoke can be characterized by the rate of change of the area of the suspected smoke region between adjacent image frames or frames, which can effectively prevent false alarms caused by stationary objects or objects with unchanged shapes, and further improve the recognition rate of smoke.
[0077] As can be understood by those skilled in the art, any suitable method known in the art can be used to perform feature accumulation and calculate the luminous flux change rate, edge length change rate or smoke area change rate, and the protection scope of the present invention is not limited to any specific method. Since these ways or methods are well known in the art, they will not be described in detail here.
[0078] If the accumulated identification feature, i.e., the rate of change of the luminous flux, the rate of change of the smoke area, or the rate of change of the edge length, is greater than the set second threshold, it is determined that smoke exists. As can be understood by those skilled in the art, the setting of the second threshold can be set by those skilled in the art according to actual needs, and the protection scope of the present invention is not limited to one or more values.
[0079] Therefore, by accumulating features of multiple consecutive images, the smoke misjudgment rate is greatly reduced, and the impact of vibration or shaking of objects in the image is significantly eliminated.
[0080] Figure 4 A processing flow chart of the background image updating module 402 in the image contrast unit according to an embodiment of the present invention is shown.
[0081] The background image updating module 402 updates the background image by the following steps:
[0082] In step 404, an image counter N is set.
[0083] In step 406, when the image counter accumulates a threshold (N) number of images, the image counter is reset, that is, the mark count of the next image is reset to i=1.
[0084] In step 408, an image with the lowest smoke probability is selected from the threshold number of images, wherein the smoke probability of the image is derived from the smoke probability obtained by the binary classification neural network in the smoke static feature recognition module 212. The image with the lowest smoke probability is then passed to the image pre-processing unit 104. The image is passed to the image pre-processing unit 104, which processes the received image into a binary image and passes the generated binary image back to the background image update module 402.
[0085] Finally, in step 410, the binary image from the image pre-processing unit is updated to a background image, and the updated background image is provided to the smoke dynamic feature recognition module 302 for generating a differential image.
[0086] Figure 5 The figure shows a processing flow chart of the image fusion module 502 in the image pre-processing unit according to an embodiment of the present invention.
[0087] At step 504, the binary image of the captured image is subtracted from the background image to obtain a difference image.
[0088] In step 506, the grayscale change function is used to calculate the grayscale change rate of the entire difference image. In one embodiment of the present invention, the image to be evaluated is obtained and grayscaled, and the vertical edge where the grayscale value of each row of the image to be evaluated decreases is determined according to the grayscale value of each row of pixels in the image to be evaluated, and the grayscale change rate of the vertical edge is calculated according to the grayscale value of the pixels contained in the vertical edge, and a variable a with an initial value of 0 is set at the same time. Each time the grayscale change rate of the vertical edge is calculated, the value of the variable a is increased by 1, and the absolute values of the grayscale change rates of the vertical edges of all rows in the image to be evaluated are summed to obtain the grayscale change rate sum S1 of the image to be evaluated, and the average value S of the grayscale change rate sum of the image to be evaluated is calculated according to the ratio of S1 and the variable a, and finally the clarity of the image to be evaluated is evaluated according to the average value S. As can be understood by those skilled in the art, the protection scope of the present invention is not limited to the above-mentioned grayscale change rate calculation method, but covers all suitable grayscale change rate calculation methods known in the art.
[0089] In step 508, the edge of the smoke portion in the captured image is obtained using a gradient function and the edge gradient change rate is determined. In one embodiment of the present invention, the edge detection algorithm may include any one of the gradient method, Roberts gradient method, Sobel algorithm, Laplace algorithm, Canny algorithm, etc. As can be understood by those skilled in the art, in addition to the above edge feature extraction method, any other suitable method may be used to extract edge features and calculate the gradient change rate. The protection scope of the present invention is not limited to the above specific gradient method, but covers all methods or methods known in the art that can be used to calculate the edge gradient change rate.
[0090] In step 510 , when the grayscale change rate is greater than the set third threshold and the edge gradient change rate is greater than the set fourth threshold, it is determined that the captured image is not clear.
[0091] In step 512, the control unit instructs the lighting unit to switch the light color according to red, yellow and blue so that the camera unit obtains three images with different colors.
[0092] In step 514, the three images of different colors are weighted and summed according to the structural similarity to obtain a fused image for image comparison. Those skilled in the art will appreciate that any suitable weighted summing method based on structural similarity known in the art can be used to perform image fusion.
[0093] Figure 6 A flow chart of an image-based smoke detection method 600 according to an embodiment of the present invention is shown.
[0094] The image-based smoke detection method 600 includes:
[0095] In step 602, an image is captured. The camera unit captures an image and transmits the captured image to the image pre-processing unit via the wireless transmission unit.
[0096] In step 604, the captured image is processed into a binary image. The image pre-processing unit generates a binary image for the smoke dynamic feature recognition module in the image comparison unit to subtract from the background image to obtain a difference image. The image pre-processing unit is also used to denoise and grayscale the captured image. Denoising helps to more accurately recognize the image, while grayscale can be used to determine whether the captured image is clear enough.
[0097] In step 606, it is determined whether there is smoke in the captured image by subtracting the binary image from the background image to extract edge features and accumulating the extracted edge features. Accumulating the extracted edge features includes subtracting a plurality of consecutive captured images from the background image to determine whether one or more of the luminous flux change rate, the smoke area change rate, and the smoke edge length change rate is greater than a set second threshold.
[0098] In step 608, the captured image is subjected to four-classification and the four-classified image is classified into smoke and no smoke, wherein the four-classification classifies the captured image into a normal lens image, a dusty lens image, a fogged lens image, and a vibrating lens image. The four-classification neural network classifies the captured image into a normal lens image, a dusty lens image, a fogged lens image, and a vibrating lens image, including giving the probabilities of the four respectively, and then the captured image enters the corresponding two-classification neural network with the maximum probability to obtain the smoke probability, and finally determines that there is smoke in the captured image when the product of the maximum probability and the smoke probability is greater than the set first threshold.
[0099] In step 610 , a smoke alarm signal is generated if it is determined that there is smoke in the captured image through both edge feature accumulation and image classification.
[0100] Embodiments of the present invention have been described above with reference to block diagrams and / or the operating instructions of methods, systems, and computer program products according to embodiments of the present invention. Each function / action noted in the frame may occur in an order different from that shown in any flow chart. For example, depending on the function / action involved, two frames shown in succession may actually be performed substantially simultaneously, or these frames may sometimes be performed in an opposite order.
[0101] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A system for image-based smoke detection, include: A camera unit for capturing images; An image pre-processing unit, which is used to process the captured image into a binary image; Image contrast unit, comprising: a smoke static feature recognition module, comprising a four-classification neural network for classifying the captured image and a corresponding two-classification neural network for classifying the classified image into smoke or no smoke, wherein the four-classification neural network classifies the captured image into a normal lens image, a dusty lens image, a fogged lens image, and a vibrated lens image; and A smoke dynamic feature recognition module, which is used to determine whether there is smoke in the captured image by subtracting the binary image from the background image to extract edge features and accumulating the extracted edge features; an image storage unit, configured to store the captured image; and A control unit controls the above units to perform corresponding operations, and generates a smoke alarm signal when both the smoke static feature recognition module and the smoke dynamic feature recognition module determine that there is smoke.
2. The system according to claim 1, It is characterized in that The system further comprises a wireless transmission unit, through which the camera unit transmits the captured image to the image pre-processing unit, and the camera unit and the wireless transmission unit together constitute a miniature camera assembly arranged in the smoke flow channel.
3. The system according to claim 2, It is characterized in that The wireless transmission unit uses at least one of Bluetooth, Wi-Fi, 3G, 4G and 5G.
4. The system according to claim 1, It is characterized in that The image pre-processing unit is also used to perform denoising and gray-scaling on the captured image.
5. The system according to claim 1, It is characterized in that Accumulating the extracted edge features includes subtracting a plurality of consecutive captured images from the background image to determine whether one or more of the luminous flux change rate, the smoke area change rate, and the smoke edge length change rate is greater than a set second threshold.
6. The system of claim 1, It is characterized in that The four-classification neural network is also used to give the possibility that the captured image is a normal lens image, a dusty lens image, a fogged lens image or a vibrating lens image, so that the captured image enters the corresponding two-classification neural network with the maximum possibility to obtain the possibility of smoke, and the smoke static feature recognition module determines that there is smoke in the captured image when the product of the maximum possibility and the possibility of smoke is greater than a set first threshold.
7. The system of claim 6, It is characterized in that The image contrast unit further includes a background image updating module, and the background image updating module updates the background image by the following steps: Set the image counter; When the image counter accumulates a threshold number of images, resetting the image counter; Selecting an image with the lowest smoke probability from among the threshold number of images, wherein the smoke probability of the image is derived from the smoke probability obtained by the binary classification neural network; and A binary image of the selected image is obtained through the image pre-processing unit and updated as the background image.
8. The system of claim 1, It is characterized in that The system further includes a lighting unit that uses a cold light source and switches light color based on whether the captured image is clear.
9. The system of claim 8, It is characterized in that The image pre-processing unit includes an image fusion module, and the image fusion module performs the following steps: subtracting the binary image of the captured image from the background image to obtain a difference image; Using a grayscale change function to calculate the grayscale change rate of the entire difference image; Using a gradient function to find the edge of the smoke portion in the captured image and determine the edge gradient change rate; Determining that the captured image is unclear when the grayscale change rate is greater than a set third threshold and the edge gradient change rate is greater than a set fourth threshold; Instructing the lighting unit to switch the light color according to red, yellow and blue to obtain three images of different colors via the control unit; as well as The three images of different colors are weightedly summed according to structural similarity to obtain a fused image for image comparison.
10. The system of claim 1, It is characterized in that The system has a standby mode and a monitoring mode, in which the camera unit captures images at a low frequency in the standby mode, and when the first smoke image is identified, the system enters a monitoring mode and sets a monitoring cycle, in which the camera unit captures images at a high frequency.
11. The system of claim 1, It is characterized in that The image storage unit classifies and saves the captured images into four categories: normal lens image, dusty lens image, fogged lens image and lens vibration image, and uploads them to the cloud platform for the training modules for the four-category neural network and the two-category neural network in the cloud platform to perform training and deep learning based on the uploaded images. The trained and learned four-category neural network and two-category neural network are downloaded to the image comparison unit.
12. A method for image-based smoke detection, include: Capture images; Processing the captured image into a binary image; Determining whether there is smoke in the captured image by subtracting the binary image from a background image to extract edge features and accumulating the extracted edge features; Performing four-classification on the captured image and classifying the image after the four-classification into smoke or no smoke, wherein the four-classification classifies the captured image into a normal lens image, a dusty lens image, a fogged lens image, and a vibrated lens image; as well as A smoke alarm signal is generated when it is determined that there is smoke in the captured image through both edge feature accumulation and image classification.
13. The method according to claim 12, It is characterized in that The captured image is transmitted using at least one of Bluetooth, Wi-Fi, 3G, 4G and 5G.
14. The method according to claim 12, It is characterized in that Accumulating the extracted edge features includes subtracting a plurality of consecutive captured images from the background image to determine whether one or more of the luminous flux change rate, the smoke area change rate, and the smoke edge length change rate is greater than a set second threshold.
15. The method according to claim 12, It is characterized in that The captured image is classified into four categories and classified into smoke-existing and smoke-free categories according to the four categories, including giving the possibility that the captured image is a normal lens image, a dusty lens image, a foggy lens image or a vibrating lens image, so that the captured image enters the corresponding two-category neural network with the maximum possibility to obtain the possibility of smoke, and the method further includes determining that there is smoke in the captured image when the product of the maximum possibility and the possibility of smoke is greater than a set first threshold.
16. The method of claim 15, It is characterized in that Further including: Set the image counter; When the image counter accumulates a threshold number of images, resetting the image counter; Selecting an image with the lowest smoke probability from among the threshold number of images, wherein the smoke probability of the image is derived from the smoke probability obtained by the binary classification neural network; and The selected image is processed into a binary image and the generated binary image is updated as the background image.
17. The method of claim 12, It is characterized in that Further including: subtracting the binary image of the captured image from the background image to obtain a difference image; Using a grayscale change function to calculate the grayscale change rate of the entire difference image; Using a gradient function to find the edge of the smoke portion in the captured image and determine the edge gradient change rate; Determining that the captured image is unclear when the grayscale change rate is greater than a set third threshold and the edge gradient change rate is greater than a set fourth threshold; Instructing the lighting unit to switch the light color according to red, yellow and blue to obtain three images with different colors; as well as The three images of different colors are weightedly summed according to their structural similarity to obtain the fused image.
18. The method of claim 12, It is characterized in that In the standby mode, images are captured at a low frequency. When the first smoke image is recognized, the monitoring mode is entered and a monitoring cycle is set. In the monitoring mode, images are captured at a high frequency.
19. The method of claim 12, It is characterized in that The method further includes classifying and saving the captured images into four categories: normal lens images, dusty lens images, fogged lens images, and lens vibration images, and uploading them to a cloud platform so that a module for neural network training in the cloud platform can perform training and deep learning based on the uploaded images.
