A method, device, apparatus and readable storage medium for detecting open flames

Abstract

The present invention provides an open flame detection method, device, equipment, and readable storage medium, relating to the technical field of computer image processing and recognition methods, including obtaining first information to obtain an open flame sample image set; traversing all open flame sample images in the open flame sample image set, and extracting single-frame images from all the open flame sample images; identifying the open flame object contained in the single-frame image based on a deep learning target detection algorithm; determining whether the single-frame image contains an open flame, and if so, obtaining and outputting an analysis result based on an optical flow method if the open flame is contained and the detection score is less than a set score threshold. The beneficial effects of the present invention are that the model adopted by the present invention has high accuracy and low false detection; the open flame detection model has been applied to multiple practical projects such as smart energy, smart construction sites, and smart parks, with high detection accuracy; and the model is lightweight.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an open flame detection method, device, equipment and a readable storage medium. Background Art

[0002] With the rise of deep learning, the design of neural networks for feature extraction and classification of flame characteristics has made automated flame detection possible. Frizzi et al. designed a simple CNN model for both feature extraction and flame classification. Some researchers have also used flame datasets to fine-tune existing network architectures, such as GoogleNet and AlexNet, for flame detection. Numerous studies have focused on improving network structures based on flame characteristics to make them more suitable for flame detection tasks. Based on video data, researchers designed a network structure combining Faster R-CNN and LSTM to classify whether a fire has occurred in a specific area within a short period of time. Researchers used a two-stage approach to detect flames in night vision. First, they used Elastic to improve the YOLOv3 network structure to increase the recognition rate of small targets. Then, they used traditional optical flow histograms for subsequent classification. Summary of the Invention

[0003] The present invention aims to provide a method, device, apparatus, and readable storage medium for detecting open flames to improve the above-mentioned problems. To achieve the above-mentioned objectives, the present invention adopts the following technical solutions:

[0004] In a first aspect, the present application provides a method for detecting an open flame, comprising:

[0005] Acquiring first information, wherein the first information is an open fire sample image set obtained by collecting an open fire sample image of the area to be detected through a collection device;

[0006] Traversing all the open fire sample images in the open fire sample image set, and extracting single-frame images from all the open fire sample images;

[0007] Based on a deep learning target detection algorithm, the open flame object contained in the single frame image is identified;

[0008] Determine whether the single-frame image contains an open flame. If so, and the detection score is less than the set score threshold, analyze the dense trajectory generated by the single-frame image based on the optical flow method, obtain the analysis result, and output it.

[0009] In a second aspect, the present application further provides an open flame detection device, comprising an acquisition module, a traversal module, an identification module, and a judgment module, wherein:

[0010] An acquisition module is configured to acquire first information, wherein the first information is an open fire sample image set obtained by acquiring an open fire sample image of a region to be detected through an acquisition device;

[0011] Traversal module: used for traversing all the open fire sample images in the open fire sample image set and extracting single-frame images from all the open fire sample images;

[0012] Recognition module: used for identifying the open flame object contained in the single frame image based on the target detection algorithm of deep learning;

[0013] Judgment module: used to determine whether the single-frame image contains an open flame. If it does and the detection score is less than the set score threshold, the dense trajectory generated by the single-frame image is analyzed based on the optical flow method, and the analysis results are obtained and output.

[0014] In a third aspect, the present application further provides an open flame detection device, comprising:

[0015] memory for storing computer programs;

[0016] A processor is configured to implement the steps of the open flame detection method when executing the computer program.

[0017] In a fourth aspect, the present application further provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned open flame detection method are implemented.

[0018] The beneficial effects of the present invention are:

[0019] 1. High model accuracy and low false detection: The open flame detection model of the present invention has been applied to multiple practical projects such as smart energy, smart construction sites, and smart parks. It has high detection accuracy and achieves almost zero false detection.

[0020] 2. The model is lightweight and can be directly deployed on edge devices: The open fire detection model of the present invention uses a GPU to train the target detection network and a CPU for dense trajectory post-processing.

[0021] 3. The target detection model has high accuracy in identifying open flames: The first-stage target detection model trained by the present invention uses a dataset with rich scenes, large quantity, and complete annotations. The improved Faster RCNN model can efficiently extract static features such as the color, shape, and texture of open flames, thereby enabling the first-stage target detection model to identify open flames with high accuracy.

[0022] 4. Dense trajectories greatly reduce the false detection rate of non-open flames: The second stage of the open flame detection model of the present invention uses a dense trajectory method based on the optical flow method to fully utilize the dynamic characteristics of open flames and the persistence of open flames in the camera, effectively eliminating the interference of luminous objects such as car lights, lighting, and neon lights, reducing the false alarm rate of the open flame detection model to zero in actual projects.

[0023] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the embodiments of the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 Schematic diagram of the open flame detection method according to an embodiment of the present invention;

[0026] Figure 2 Schematic diagram of the structure of the open flame detection device according to an embodiment of the present invention;

[0027] Figure 3 Schematic diagram of the structure of the open flame detection equipment described in an embodiment of the present invention.

[0028] In the figure: 701, acquisition module; 702, traversal module; 7021, acquisition unit; 7022, determination unit; 7023, confirmation unit; 7024, selection unit; 703, recognition module; 7031, input unit; 7032, judgment result unit; 7033, information acquisition unit; 7034, sub-graph acquisition unit; 704, judgment module; 7041, first recognition unit; 7042, first judgment unit; 7043, retrieval unit; 7044, first acquisition unit; 7045, generation unit; 7046, first extraction unit; 70461, scaling unit; 70462, second extraction unit; 70463, calculation unit; 7047, second judgment unit; 7048, third judgment unit; 800, open flame detection equipment; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are used only to distinguish the description and should not be understood as indicating or implying relative importance.

[0031] Example 1:

[0032] This embodiment provides a method for detecting open flames.

[0033] In existing technologies, flame detection technology based on traditional machine learning first performs preprocessing operations such as denoising, data enhancement, and threshold segmentation on real-time monitoring video images. It then extracts static and dynamic features of the flame, which are then fed into classification algorithms such as support vector machines, random forests, and Bayesian classifiers for flame recognition. Traditional machine learning-based flame detection technology requires manual extraction of flame features. Without a comprehensive feature description system, the quality of feature extraction directly affects flame detection results.

[0034] At present, the difficulties of open flame detection projects are mainly reflected in two aspects: first, the color of light, night car lights, neon lights, etc. is very similar to that of flames, which greatly interferes with flame detection and causes a high false detection rate; second, flame movement is more random than the rigid movement or flexible distortion of general targets. Changes in fire intensity or external forces such as wind direction drive the flame to grow and change its shape, affecting the accuracy of detection.

[0035] To address this issue, the flame detection model designed in this paper is a two-stage model consisting of an object detection model and dense trajectory post-processing. The object detection network designed in the first stage effectively, efficiently, and robustly extracts static features such as flame shape, color, and texture, achieving a recall rate of 95.8% and an AP of 88.0%. However, in actual project testing, sunlight, neon lights, and nighttime car lights were sometimes mistakenly detected as flames.

[0036] To eliminate false alarms of open flames, we take into account the dynamic characteristics of flames and use N consecutive frames of images to generate a three-dimensional fire-tube. Dense points are sampled in each frame of the fire-tube, and the trajectories of these densely sampled points at multiple spatial scales are tracked based on the displacement information generated by the dense optical flow field.

[0037] It should be noted that the target detection algorithm used in the present invention is the Faster RCNN detection method, which is a typical representative of the two-stage target detection model.

[0038] See also Figure 1 , the figure shows that the method includes step S100, step S200, step S300 and step S400.

[0039] S100: Acquire first information, where the first information is an open fire sample image of a to-be-detected area collected by an acquisition device to obtain an open fire sample image set.

[0040] As you can understand, in this step, based on the deployment scenario, the requirements for real-time flame detection, high recall, and low false detection, we selected a two-stage object detection algorithm based on a modified Faster RCNN for open flame detection. The image stream from the camera is fed into our trained object detection model to generate image inference results, including the open flame category, the area marked with a rectangular box to indicate the open flame, and the open flame score.

[0041] It should be noted that the acquisition device can be a camera. Deep learning-based object detection algorithms are mainly divided into two categories: two-stage and one-stage. Common two-stage object detection algorithms include R-CNN, SPP-Net, Fast R-CNN, and Faster R-CNN. Common one-stage object detection algorithms include SSD, RetinaNet, YOLOv3, YOLOv5, and PP-YOLOv2.

[0042] S200 , traverse all the open fire sample images in the open fire sample image set, and extract single-frame images from all the open fire sample images.

[0043] It can be understood that, in this step, S200 includes S201, S202, S202 and S204, wherein:

[0044] S201, collecting a video of an open fire in a to-be-detected area by a collection device, and extracting an image to be identified from the video of the open fire, wherein the image to be identified includes a luminous object that appears in a plurality of consecutive frames of images in a temporal sequence;

[0045] S202, determining a duration for which the luminous object appears continuously in the multiple frames of images, and a moving distance of the luminous object within the duration for which the luminous object appears continuously in the multiple frames of images;

[0046] S203: when the duration of the continuous appearance of the luminous object is greater than a preset duration, and / or the moving distance of the luminous object within the duration of the continuous appearance is less than a preset displacement, confirming that the luminous object is an open flame;

[0047] S204: Select a single-frame image of the open flame within the duration of the continuous appearance.

[0048] In some possible implementations, a luminous object that is identified for a duration greater than or equal to a preset duration is not a false alarm, but an object that appears with a high probability. For example, in the scenario of identifying an explosion and open flame, if the duration of the luminous object's continuous appearance is greater than or equal to a preset duration, it means that the image is most likely an open flame image, indicating that the acquisition scene of the image to be identified is a scene where an open flame has occurred; and / or, a movement distance less than a preset displacement indicates that the luminous object occurs at a fixed position. For example, in the scenario of identifying an explosion and open flame, since car lights and open flame images are similar, if a luminous object is identified and the movement distance of the luminous object is large, it means that the identified luminous object is not an open flame image, but may be a movable luminous object such as a car light; if a luminous object is identified and the movement distance of the luminous object is small, it means that the identified luminous object is most likely an open flame image; in this case, determining the acquisition scene of the image to be identified as an open flame scene, that is, confirming that the luminous object is an open flame, can reduce false alarms caused by using a single-frame image for recognition, and can improve the accuracy of the recognition results.

[0049] S300: Identify the open flame object contained in the single-frame image using a deep learning-based target detection algorithm.

[0050] It is understandable that in this step S300, the previous steps also include S301, S302, S303 and S304, wherein:

[0051] S301: Inputting a captured single-frame image into a preset target detection model through a capture device to obtain an open fire detection result, wherein the open fire detection result includes an open fire category, an area marked with an open fire using a rectangular frame, and an open fire score;

[0052] S302: Determine whether the score of the single-frame image in the open fire detection result is greater than a set score threshold; if not, obtain a video stream starting from the single-frame image and N consecutive frames of images;

[0053] S303: Acquire second information, where the second information includes acquiring the position and size of the single-frame image;

[0054] S304 , obtaining sub-images of the same size at the position of the single-frame image of the consecutive N frames of images, and generating a three-dimensional fire tube from the sub-images in sequence.

[0055] If no open flame is found in the single-frame image in the above steps, the open flame detection recognition result is not derived; if an open flame is found in a single-frame image and the score is greater than the set score threshold thre, the open flame detection recognition result is directly derived; if an open flame is found in a single-frame image and the score is less than the set score threshold thre, a continuous video stream is taken from that frame to form a fire-tube, and the dense trajectory generated by the fire-tube based on the optical flow method is analyzed. The trajectory analysis model is used to classify the open flame and non-open flame (false detection) in the fire-tube, and finally the result is output.

[0056] S400: Determine whether the single-frame image contains an open flame. If so, and the detection score is less than a set score threshold, analyze the dense trajectory generated by the single-frame image based on the optical flow method, obtain the analysis result, and output it.

[0057] It can be understood that the steps before step S400 include S401, S402, S403, S404 and S405, wherein:

[0058] S401, identifying an open flame object contained in the single-frame image based on a target detection algorithm;

[0059] S402: Determine whether the single-frame image contains an open flame. If so, score the single-frame image according to a preset scoring rule to obtain a scoring result.

[0060] S403: when it is detected that the scoring result is less than a threshold, retrieving a video stream after the single-frame picture from the single-frame picture;

[0061] S404, obtaining continuous frame image information in the video stream;

[0062] S405 : Based on the target detection algorithm, the continuous frame image information is processed according to preset rules to generate a three-dimensional model.

[0063] It should be noted that the process is divided into three steps. The first step is fire tube generation. When an open fire is detected in a frame and its score is less than a threshold, thre, the video stream is extracted starting from that frame, generating N consecutive frames. Based on the position and size of the rectangular box enclosing the open fire area inferred by the object detection model in step 1, sub-images of the same size at the same position in N consecutive frames are obtained. These N sub-images are then combined in their original order to generate a 3D fire tube.

[0064] Furthermore, this step S400 includes S406, S407 and S408, wherein:

[0065] S406, extracting the three-dimensional model according to the optical flow method and a preset visual library to obtain a dense optical flow field;

[0066] S407: Based on the trajectories of the tracking points extracted in the dense optical flow field at different spatial scales, determine whether to retain the trajectory generated by each feature point in the fire tube image set in the three-dimensional model according to the Shi-Tomasi corner detection method;

[0067] S408. Count the number of the trajectories and determine whether the total number of the trajectories is lower than the quantity threshold; if so, delete them; if not, use the target detection model to determine whether there is an open flame in the consecutive frames after the single-frame image; if so, update the open flame score and output the open flame detection result.

[0068] It should be noted that the second step is to calculate based on the optical flow method, where optical flow is the instantaneous speed of the pixel movement of a moving object in space on the observation imaging plane. Based on the two assumptions of constant brightness and continuous time or "small motion", the basic constraint equation of optical flow is I(x,y,t)=I(x+dx,y+dy,t+dt)

[0069] Where I(x,y,t) is the pixel intensity at (x,y) at time t, dt is the interval between two adjacent frames, and the distance moved by the pixel (x,y) during dt is (dx,dy), where dx is the distance moved along the x-axis during dt, and dy is the distance moved along the y-axis during dt.

[0070] Perform Taylor expansion on the above constraint equation and divide it by dt to obtain I x u+I y v+I t =0, where I x ,I y ,I t Both are obtained from image data. u and v are the velocity vectors of the optical flow along the x-axis and y-axis respectively, which are unknown quantities.

[0071] We use the OpenCV (computer vision library) provided by The interface extracts dense optical flow from the generated fire-tube. The effect can be seen in the animated image. In the dense optical flow field, the position of the tracking point in the next frame can be determined by the formula in the dense trajectory.

[0072] Furthermore, the above step S406 then includes S4061, S4062 and S4063, wherein:

[0073] S4061, scaling the images in the fire tube image set according to a preset ratio and sampling them using a sliding window at each spatial scale to obtain densely sampled feature points;

[0074] S4062, extracting feature points of any image in the fire tube image set as points to be detected;

[0075] S4063. Using the median filtering method, obtain the position of the feature point corresponding to the next picture of any of the pictures and the positions of the feature points corresponding to all pictures, and calculate the trajectory of each sampled feature point at different spatial scales based on the obtained positions of the corresponding feature points.

[0076] It should be noted that the third step is dense trajectory calculation, which sets a fixed scaling ratio to obtain several different spatial scales.

[0077] Suppose a feature point of the tth frame is: P t =(x t ,y t ), after the optical flow method is processed, the dense optical flow field w=(u t , v t ) uses median filtering to obtain the position P of the corresponding feature point in the t+1th frame t+1 , the formula is:

[0078]

[0079] Where M is the median filter kernel, Yes (x t ,y t ) rounded position coordinates. Scale the fire-tube image according to the set ratio and sample it at each spatial scale. According to the above formula, the positions of the corresponding feature points of all frames are obtained. Each sampled feature point obtains a trajectory at different spatial scales: (P t ,P t+1 ,P t+2 ,...).

[0080] We use the Shi-Tomasi corner detection method to determine whether to keep a certain track. The autocorrelation matrix of each sampling feature point in the track is:

[0081]

[0082] Where g x is the partial derivative of the pixel in the x-axis direction, g yis the partial derivative of the pixel point in the y-axis direction. Compare the minimum eigenvalue of matrix G with the set threshold. If the minimum eigenvalue is lower than the threshold, the feature point is not included in the trajectory. Only when the minimum eigenvalue is higher than the threshold, the feature point is saved. Only when the length of the trajectory meets the set length L, the trajectory is saved.

[0083] It should be noted that when the number of trajectories generated by the fire-tube after passing through the dense trajectories is lower than the number threshold T, it is considered that there is no flame in this frame and no prediction result is deduced; if the number of trajectories generated by the fire- is not lower than the number threshold T, use the object detection model in step1 to judge whether there is open fire in the subsequent continuous F (F < N / 2) frames. If there is open fire in all of them, update the open fire score and deduce the result of open fire detection, otherwise do not deduce.

[0084] Embodiment 2:

[0085] As Figure 2 shown, this embodiment provides an open fire detection device. Refer to Figure 2 The device includes an acquisition module 701, a traversal module 702, an identification module 703 and a judgment module 704, where:

[0086] The acquisition module 701: is used to acquire the first information, and the first information is to acquire the open fire sample images of the area to be detected through the acquisition device to obtain an open fire sample image set;

[0087] The traversal module 702: is used to traverse all the open fire sample images in the open fire sample image set and extract single-frame pictures from all the open fire sample images;

[0088] The identification module 703: is used to identify the open fire objects included in the single-frame pictures based on the object detection algorithm of deep learning;

[0089] The judgment module 704: is used to judge whether the single-frame pictures contain open fire. If it contains and the detection score is less than the set score threshold, analyze the dense trajectories generated by the single-frame pictures based on the optical flow method, obtain the analysis result and output it.

[0090] Further, before the judgment module 704, there are a first identification unit 7041, a first judgment unit 7042, a retrieval unit 7043, a first acquisition unit 7044 and a generation unit 7045, where:

[0091] The first identification unit 7041: is used to identify the open fire objects included in the single-frame pictures based on the object detection algorithm;

[0092] The first judging unit 7042 is configured to judge whether the single-frame image contains an open flame, and if so, to score the single-frame image according to a preset scoring rule to obtain a scoring result;

[0093] The retrieving unit 7043 is configured to retrieve the video stream after the single-frame picture when detecting that the scoring result is less than a threshold;

[0094] The first acquiring unit 7044 is configured to acquire continuous frame picture information in the video stream;

[0095] The generating unit 7045 is configured to process the continuous frame image information according to preset rules based on the target detection algorithm to generate a three-dimensional model.

[0096] Furthermore, the judgment module 704 includes a first extraction unit 7046, a second judgment unit 7047 and a third judgment unit 7048, wherein:

[0097] The first extraction unit 7046 is configured to extract the three-dimensional model according to the optical flow method and a preset visual library to obtain a dense optical flow field;

[0098] A second judgment unit 7047 is configured to determine whether to retain the trajectory generated by each feature point in the fire tube image set in the three-dimensional model based on the trajectories of the tracking points extracted in the dense optical flow field at different spatial scales and using the Shi-Tomasi corner detection method;

[0099] The third judgment unit 7048 is used to count the number of the trajectories and determine whether the total number of the trajectories is lower than the quantity threshold; if so, delete them; if not, use the target detection model to determine whether there is an open flame in the consecutive frames after the single-frame image; if so, update the open flame score and output the open flame detection result.

[0100] Furthermore, the first extraction unit 7046 then includes a scaling unit 70461, a second extraction unit 70462, and a calculation unit 70463, wherein:

[0101] Scaling unit 70461: used to scale the images in the fire tube image set according to a preset ratio and perform sampling using a sliding window at each spatial scale to obtain densely sampled feature points;

[0102] The second extraction unit 70462 is used to extract feature points of any image in the fire tube image set as points to be detected;

[0103] Calculation unit 70463: used to obtain the position of the feature point corresponding to the next picture of any of the pictures and the position of the feature points corresponding to all pictures by using the median filtering method, and calculate the trajectory of each sampled feature point at different spatial scales based on the obtained position of the corresponding feature point.

[0104] Furthermore, the traversal module 702 includes a collection unit 7021, a determination unit 7022, a confirmation unit 7023 and a selection unit 7024, wherein:

[0105] The acquisition unit 7021 is configured to acquire a video of an open fire in a to-be-detected area through an acquisition device, and extract an image to be identified from the video of the open fire, wherein the image to be identified includes a luminous object that appears in a plurality of consecutive frames of images in a temporal sequence;

[0106] Determining unit 7022: used to determine the duration of the continuous appearance of the luminous object in the multiple frames of images, and the moving distance of the luminous object within the duration of the continuous appearance of the luminous object in the multiple frames of images;

[0107] Confirmation unit 7023: configured to confirm that the luminous object is an open flame when the duration of continuous appearance of the luminous object is greater than a preset duration and / or the moving distance of the luminous object within the duration of continuous appearance is less than a preset displacement;

[0108] The selection unit 7024 is used to select a single frame image where the open flame is located within the duration of the continuous appearance.

[0109] Furthermore, the recognition module 703 includes an input unit 7031, a judgment result unit 7032, an information acquisition unit 7033 and a sub-image acquisition unit 7034, wherein:

[0110] Input unit 7031: used to input the single-frame image collected by the collection device into a preset target detection model to obtain an open fire detection result, which includes the open fire category, the area marked with the open fire using a rectangular box, and the open fire score;

[0111] The result determination unit 7032 is configured to determine whether the score of the single-frame image in the open fire detection result is greater than a set score threshold. If not, the video stream starting from the single-frame image and N consecutive frames of images are obtained.

[0112] The information acquisition unit 7033 is configured to acquire second information, where the second information includes the position and size of the single-frame image.

[0113] The sub-image acquisition unit 7034 is configured to acquire sub-images of the same size at the positions of the N consecutive frames of images, and generate a three-dimensional fire tube using the sub-images in sequence.

[0114] It should be noted that, regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0115] Example 3:

[0116] Corresponding to the above method embodiment, this embodiment further provides an open flame detection device. The open flame detection device described below and the open flame detection method described above can refer to each other.

[0117] Figure 3 A block diagram of an open flame detection device 800 is shown according to an exemplary embodiment. The open flame detection device 800 may include: a processor 801, a memory 802. The open flame detection device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0118] The processor 801 is used to control the overall operation of the open flame detection device 800 to complete all or part of the steps in the open flame detection method described above. The memory 802 is used to store various types of data to support the operation of the open flame detection device 800. This data may include, for example, instructions for any application or method operating on the open flame detection device 800, as well as application-related data, such as contact information, sent and received messages, images, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules. The above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the open flame detection device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G or 4G, or a combination of one or more thereof, so the corresponding communication component 805 may include: a Wi-Fi module, a Bluetooth module, an NFC module.

[0119] In an exemplary embodiment, the open flame detection device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned open flame detection method.

[0120] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the above-described open flame detection method. For example, the computer-readable storage medium may be the aforementioned memory 802 including the program instructions. The program instructions may be executed by the processor 801 of the open flame detection device 800 to implement the above-described open flame detection method.

[0121] Example 4:

[0122] Corresponding to the above method embodiment, this embodiment further provides a readable storage medium. The readable storage medium described below and the open flame detection method described above can refer to each other.

[0123] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the open flame detection method of the above method embodiment.

[0124] The readable storage medium may specifically be any readable storage medium that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0125] In summary, the advantages of the present invention are high model accuracy and low false detection: the open flame detection model of the present invention is applied to multiple actual projects such as smart energy, smart construction sites and smart parks, with high detection accuracy and almost zero false detection. The model is lightweight and can be directly deployed on edge devices; the open flame detection model of the present invention uses a GPU to train the target detection network and uses a CPU for dense trajectory post-processing. The target detection model has high accuracy in identifying open flames: the data set used by the first-stage target detection model trained by the present invention is rich in scenes, large in number and well-labeled. The improved Faster RCNN model can efficiently extract static features such as color, shape, and texture of open flames, thereby making the first-stage target detection model have high accuracy in identifying open flames. Dense trajectories greatly reduce the false detection rate of non-open flames; the second-stage method of generating dense trajectories based on the optical flow method of the open flame detection model of the present invention fully utilizes the dynamic characteristics of open flames and the persistence of open flames in the camera, effectively eliminating the interference of luminous objects such as car lights, lighting, and neon lights, so that the false alarm rate of the open flame detection model in actual projects is reduced to 0.

[0126] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0127] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for detecting open flames, characterized in that: include: Acquiring first information, wherein the first information is an open fire sample image set obtained by collecting an open fire sample image of a to-be-detected area through a collection device; Traversing all the open fire sample images in the open fire sample image set, and extracting single-frame images from all the open fire sample images; Identify the open flame object contained in the single-frame image using a deep learning-based target detection algorithm; Determine whether the single-frame image contains an open flame. If so, and the detection score is less than a set score threshold, analyze the dense trajectory generated by the single-frame image based on the optical flow method, obtain the analysis result, and output it; The traversing all the open fire sample images in the open fire sample image set and extracting single-frame images from all the open fire sample images includes: S201, collecting a video of an open fire in a to-be-detected area by a collection device, and extracting an image to be identified from the video of the open fire, wherein the image to be identified includes a luminous object that appears in a plurality of consecutive frames of images in a temporal sequence; S202, determining a duration for which the luminous object appears continuously in the multiple frames of images, and a moving distance of the luminous object within the duration for which the luminous object appears continuously in the multiple frames of images; S203: when the duration of the continuous appearance of the luminous object is greater than a preset duration, and / or the moving distance of the luminous object within the duration of the continuous appearance is less than a preset displacement, confirming that the luminous object is an open flame; S204: Select a single-frame image of the open flame within the duration of the continuous appearance.

2. The open flame detection method according to claim 1, characterized in that: If the detection score is less than the set score threshold, the dense trajectory generated by the single frame image is analyzed based on the optical flow method, and the analysis result is obtained and output, which includes: Identifying the open flame object contained in the single-frame image based on a target detection algorithm; Determining whether the single-frame image contains an open flame, and if so, scoring the single-frame image according to a preset scoring rule to obtain a scoring result; When it is detected that the scoring result is less than a threshold, the video stream after the single frame picture is retrieved from the single frame picture; Obtaining continuous frame image information in the video stream; Based on the target detection algorithm, the continuous frame image information is processed according to preset rules to generate a three-dimensional model.

3. The open flame detection method according to claim 2, characterized in that: If the detection score is less than the set score threshold, the dense trajectory generated by the single frame image is analyzed based on the optical flow method to obtain and output the analysis results, including: Extracting the three-dimensional model according to the optical flow method and a preset visual library to obtain a dense optical flow field; determining whether to retain the trajectory generated by each feature point in the fire tube image set in the three-dimensional model based on the Shi-Tomasi corner detection method based on the trajectories of the tracking points extracted in the dense optical flow field at different spatial scales; Count the number of the trajectories and determine whether the total number of the trajectories is lower than the quantity threshold; if so, delete them; if not, use the target detection model to determine whether there is an open flame in the consecutive frames after the single-frame image. If so, update the open flame score and output the open flame detection result.

4. The open flame detection method according to claim 3, characterized in that: The three-dimensional model is extracted to obtain a dense optical flow field, and then the following steps are included: Scaling the images in the fire tube image set according to a preset ratio and sampling them using a sliding window at each spatial scale to obtain densely sampled feature points; Extracting feature points of any image in the fire tube image set as points to be detected; The median filtering method is used to obtain the position of the feature point corresponding to the next picture of any of the pictures and the position of the feature points corresponding to all pictures. According to the obtained positions of the corresponding feature points, the trajectory of each sampled feature point at different spatial scales is calculated.

5. An open flame detection device, characterized in that: include: Acquisition module: used to acquire the first information, which is the open fire in the area to be detected collected by the acquisition device Sample image, obtain the open fire sample image set; Traversal module: used for traversing all the open fire sample images in the open fire sample image set and extracting single-frame images from all the open fire sample images; Recognition module: used for identifying the open flame object contained in the single frame image based on the target detection algorithm of deep learning; A judgment module is used to determine whether the single-frame image contains an open flame. If so, and the detection score is less than a set score threshold, the dense trajectory generated by the single-frame image is analyzed based on the optical flow method, and the analysis result is obtained and output; The traversing all the open fire sample images in the open fire sample image set and extracting single-frame images from all the open fire sample images includes: S201, collecting a video of an open fire in a to-be-detected area by a collection device, and extracting an image to be identified from the video of the open fire, wherein the image to be identified includes a luminous object that appears in a plurality of consecutive frames of images in a temporal sequence; S202, determining a duration for which the luminous object appears continuously in the multiple frames of images, and a moving distance of the luminous object within the duration for which the luminous object appears continuously in the multiple frames of images; S203: when the duration of the continuous appearance of the luminous object is greater than a preset duration, and / or the moving distance of the luminous object within the duration of the continuous appearance is less than a preset displacement, confirming that the luminous object is an open flame; S204: Select a single-frame image of the open flame within the duration of the continuous appearance.

6. The open flame detection device according to claim 5, characterized in that: The judgment module previously includes: A first recognition unit is configured to recognize the open flame object contained in the single frame image based on a target detection algorithm; A first judging unit is configured to judge whether the single-frame image contains an open flame, and if so, to score the single-frame image according to a preset scoring rule to obtain a scoring result; A retrieving unit: configured to retrieve the video stream after the single-frame picture when detecting that the scoring result is less than a threshold; A first acquisition unit is used to acquire continuous frame image information in the video stream; A generation unit is used to process the continuous frame image information based on the target detection algorithm and according to preset rules to generate a three-dimensional model.

7. The open flame detection device according to claim 6, characterized in that: The judgment module includes: A first extraction unit is configured to extract the three-dimensional model according to an optical flow method and a preset visual library to obtain a dense optical flow field; A second judgment unit is configured to judge whether to retain the trajectory generated by each feature point in the fire tube image set in the three-dimensional model based on the trajectories of the tracking points extracted in the dense optical flow field at different spatial scales and according to the Shi-Tomasi corner detection method; A third judgment unit is configured to count the number of the trajectories and determine whether the total number of the trajectories is lower than a quantity threshold; If yes, delete it; if not, use the target detection model to determine whether there is an open fire in the consecutive frames after the single frame image. If so, update the open fire score and output the open fire detection result.

8. The open flame detection device according to claim 7, characterized in that: The first extraction unit then comprises: Scaling unit: used for scaling the images in the fire tube image set according to a preset ratio and sampling using a sliding window at each spatial scale to obtain densely sampled feature points; A second extraction unit is used to extract feature points of any image in the fire tube image set as points to be detected; Calculation unit: used to obtain the position of the feature point corresponding to the next picture of any of the pictures and the position of the feature points corresponding to all pictures by using the median filtering method, and calculate the trajectory of each sampled feature point at different spatial scales based on the obtained position of the corresponding feature point.

9. An open flame detection device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the open flame detection method according to any one of claims 1 to 4 when executing the computer program.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the open flame detection method according to any one of claims 1 to 4.

Images