Deep learning fire detection system using IR camera-based image preprocessing

Abstract

The present invention relates to a fire detection system and includes an image preprocessing unit that preprocesses an IR image, which is an image captured by an IR camera, to define a fire interest region, and a deep learning unit that determines whether there is a fire based on a deep learning model that takes the fire interest region defined in the image preprocessing unit as input. According to the present invention, the problem of false positives and missed detections caused by mistaking high-temperature equipment or lighting devices for flames is minimized, and even irregular and small initial flames can be precisely recognized.

Description

Technology Field

[0001] The present invention relates to fire detection technology, and more specifically, to fire detection technology that performs real-time monitoring based on an IR (Infrared) camera. Background Technology

[0003] Fires occurring in various indoor and outdoor environments, such as underground parking lots, industrial sites, and enclosed spaces, are accompanied by multiple signals including heat, smoke, and flames. However, because they often manifest in subtle or irregular forms in their early stages, relying solely on conventional sensor-based fire detectors—such as heat, smoke, and flame detectors—has limitations in early detection. In particular, in dark environments or spaces with restricted airflow, the diffusion of smoke and heat is delayed, slowing down the detection response speed and making it difficult to respond effectively in the initial stages of a fire. While some systems are currently attempting detection using thermal imaging cameras, these methods remain based on threshold-based analysis that detects pixels above a certain temperature. Consequently, they simultaneously suffer from false positives—misidentifying non-fire heat sources like high-temperature equipment or lighting as flames—and misses—failing to detect small, early flames. Furthermore, the lack of the capability to distinguish between static high-temperature areas and actual flames makes it difficult to respond reliably to diverse environments.

[0004] Conventional thermal imaging-based fire detection systems rely on a simple threshold-based method that detects high-temperature areas based only on pixels exceeding a certain temperature. Consequently, non-fire heat sources originating from high-temperature sources, such as lighting fixtures or industrial equipment, are frequently mistaken for flames, and initial flames with irregular fluid forms are not recognized.

[0005] Most existing fire detection systems rely on a single temperature threshold or flame shape, leading to reliability issues where high-temperature objects are misidentified as flames or actual flames are not detected; furthermore, in complex environments, both false positives and false negatives occur simultaneously. Prior art literature

[0006] Republic of Korea Published Patent 10-2010-0127583 The problem to be solved

[0007] The present invention was devised to solve the above-mentioned problems, and aims to provide a fire detection system that accurately identifies a flame by applying four image preprocessing techniques—such as high temperature region, flame size (area), flame shape, and flame mobility—to a thermal image acquired from an IR camera to precisely extract a fire interest region, and inputting this into a deep learning-based object detection model.

[0008] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0010] The present invention for achieving such an objective relates to a fire detection system and includes an image preprocessing unit that preprocesses an IR image, which is an image captured by an IR camera, to define a fire interest region, and a deep learning unit that determines whether there is a fire based on a deep learning model that takes the fire interest region defined in the image preprocessing unit as input.

[0011] The image preprocessing unit may comprise: a high-temperature region extraction unit that extracts high-temperature regions that are above a reference temperature in the IR image; a size analysis unit that removes the remaining high-temperature regions from the high-temperature regions extracted by the high-temperature region extraction unit, leaving only high-temperature regions that are above a predetermined reference size; a shape analysis unit that recognizes the flame shape by analyzing the outer shape and internal pattern of the remaining high-temperature regions through the size analysis unit; and a mobility analysis unit that analyzes the movement path of the flame recognized by the shape analysis unit to distinguish between a static heat source and a dynamic flame, and defines the dynamic flame region as a fire interest region.

[0012] The above high-temperature region extraction unit can extract the high-temperature region by performing a binarization process that converts the high-temperature region to white and the remaining region to black, and by extracting the outer contour of the high-temperature region.

[0013] The shape analysis unit described above can analyze the shape by detecting corner points in high-temperature regions using the Harris corner detection technique. In this case, the shape analysis unit can recognize a shape as a flame if the number of corner points exceeds a predetermined threshold.

[0014] The above deep learning unit can determine whether there is a fire based on the YOLO v8 Nano model. Effects of the invention

[0016] According to the present invention, the problem of false positives and missed detections caused by mistaking high-temperature equipment or lighting devices for flames is minimized, and even irregular and small initial flames can be precisely recognized.

[0017] In addition, according to the present invention, unnecessary background information is removed through a preprocessing process, and the deep learning model is made to use only candidate regions where actual flames are likely to occur as targets for training and inference, thereby improving the detection accuracy and reliability of the entire system. In particular, image-based preprocessing can reflect various characteristics of flames, which differentiates it from existing simple thermal imaging systems or AI-based flame detection systems, and has the advantage of acting as a key element that enables real-time high-precision fire detection even in complex and diverse environments. Brief explanation of the drawing

[0019] FIG. 1 is a block diagram schematically illustrating the configuration of a fire detection system according to one embodiment of the present invention. Figure 2 is an image taken with an IR camera. Figure 3 illustrates the extraction of a high-temperature region from an image captured by an IR camera in the present invention. Figure 4 illustrates the maximum area extracted from the high-temperature region in the present invention. Figure 5 is a binarized image of an image captured by an IR camera in the present invention. Figure 6 shows the results of the analysis of the size ratio of the flame area among the total pixels in the present invention. Figure 7 illustrates the shape analysis results in the present invention. Figure 8 illustrates the analysis of flame movement over time in the present invention. Figure 9 visually illustrates the movement paths of flame center points in the present invention. Figure 10 illustrates the flame detection analysis results using deep learning in the present invention. Figure 11 is a graph showing the results of fire detection analysis using deep learning in an experiment of the present invention. Specific details for implementing the invention

[0020] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.

[0021] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0022] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0023] Furthermore, in the description referring to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. In describing the present invention, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the present invention, such detailed description is omitted.

[0024] The present invention relates to a fire detection system.

[0025] FIG. 1 is a block diagram schematically illustrating the configuration of a fire detection system according to an embodiment of the present invention. In FIG. 1, the IR camera (10) is a camera that detects and captures infrared rays.

[0026] Referring to FIG. 1, the fire detection system includes an image preprocessing unit (100) and a deep learning unit (200).

[0027] The image preprocessing unit (100) preprocesses the IR image, which is an image captured by the IR camera (10), to define the Fire Region of Interest (F-ROI).

[0028] The deep learning unit (200) determines whether there is a fire based on a deep learning model that takes a fire interest region (F-ROI) defined in the image preprocessing unit (100) as input.

[0029] The image preprocessing unit (100) comprises a high-temperature region extraction unit (110) that extracts a high-temperature region that is above a reference temperature in an IR image, a size analysis unit (120) that removes the remaining high-temperature regions while keeping only the high-temperature regions that are above a predetermined reference size among the high-temperature regions extracted by the high-temperature region extraction unit (100), a shape analysis unit (130) that recognizes the flame shape by analyzing the outer shape and internal pattern of the remaining high-temperature regions through the size analysis unit (120), and a mobility analysis unit (140) that analyzes the movement path of the flame recognized by the shape analysis unit (130) to distinguish between a static heat source and a dynamic flame, and defines the dynamic flame region as a fire interest region.

[0030] In one embodiment of the present invention, the high-temperature region extraction unit (110) can extract the high-temperature region by performing a binarization process that converts the high-temperature region to white and the remaining region to black, and by extracting the outer contour of the high-temperature region.

[0031] The shape analysis unit (140) can analyze the shape by detecting corner points in the high-temperature region using the Harris corner detection technique. At this time, the shape analysis unit (140) can recognize the shape as a flame shape if the number of corner points is greater than or equal to a predetermined threshold.

[0032] The deep learning unit (200) can determine whether there is a fire based on the YOLO v8 Nano model.

[0034] The present invention proposes a system that infers a fire interest region by applying four image preprocessing techniques—temperature, size, shape, and movement—based on thermal images collected through an IR camera, and then precisely detects flames using a deep learning object detection algorithm such as YOLO v8. This system can simultaneously improve the problem of false positives, where non-fire heat sources generated from high-temperature equipment or lighting fixtures are mistaken for flames, and the problem of missing detections, where small or irregular initial flames are not recognized.

[0035] To this end, the fire monitoring system of the present invention first filters flame candidates based on high-temperature regions for a thermal image input from an IR camera (10), then removes small heat sources through area analysis, extracts unique structural characteristics of the flame through Harris corner detection to more accurately determine whether the flame actually exists, and then performs image preprocessing to distinguish between static heat sources and actual flames by performing mobility analysis based on the characteristic of the flame center moving over time, and to infer the accuracy of fire occurrence.

[0036] In the fire monitoring system of the present invention, the preprocessed image is defined as a fire interest region and input into a deep learning model such as YOLO v8 nano, thereby detecting the fire in a way that accurately and reliably detects the presence of flames in real time.

[0037] Temperature is one of the most important factors in fire detection and analysis. Fires generate high temperatures, causing the temperature of the surrounding environment to rise rapidly. An IR camera (10) is effective in detecting these temperature changes and visually representing the flame area. The IR camera (10) detects the heat of an object in the infrared region to generate an image and has the characteristic that the color changes depending on the temperature of the object. Generally, objects with high temperatures are represented in bright colors (colors close to white), which makes it easy to identify flames or high-temperature objects.

[0038] Figure 2 is an image taken with an IR camera.

[0039] Figure 2 is an actual thermal image captured with an IR camera, showing the characteristic that when a flame occurs, the high-temperature region appears in a brighter color than the surroundings.

[0040] In the IR image captured by the IR camera (10), the high-temperature region appears as a relatively bright color, so the high-temperature region extraction unit (110) first detects the high-temperature region through a binarization process. Binarization is performed by converting the high-temperature region to white (255) and the remaining region to black (0), and then the outer contours are extracted using the find contours function. In the IR image, the brightness within the image increases as the temperature increases, and in particular, the region that is close to white represents the highest temperature flame region. The detected contours represent the shape of the high-temperature region, and based on this, a bounding box is drawn on the original image to visually emphasize the high-temperature region. In this process, the draw contours function is used to emphasize the outline for each contour.

[0041] Figure 3 illustrates the extraction of a high-temperature region from an image captured by an IR camera in the present invention.

[0042] Figure 3 shows the result of visually highlighting the high-temperature region, with the white area in the center corresponding to the high-temperature flame and visualized with a green outline.

[0043] Since multiple high-temperature regions may be detected in the extracted high-temperature region image, a process is required to selectively detect only the region corresponding to the largest high-temperature region.

[0044] The size analysis unit (120) calculates the area of ​​each contour using a contour area function, which calculates the area based on the number of pixels included in a given contour. Then, a Max function is used to select the contour with the largest area among several contours. The Max function extracts the contour with the largest value among the area values ​​calculated by the contour area function, thereby identifying the major high-temperature area where flames are most likely to exist.

[0045] This process is intended to filter out high-temperature areas generated by small and temporary heat sources, such as lighters or candles, and to leave only the areas that are highly likely to develop into actual fires. The size analysis unit (120) stores the contour with the largest area in the largest contour variable and excludes the remaining small high-temperature areas from the analysis.

[0046] Figure 4 illustrates the maximum area extracted from the high-temperature region in the present invention.

[0047] Figure 4 is an image visually representing the result of the largest contour selected by the Max function, where the main area where the actual flame occurred is highlighted with a green contour.

[0048] Figure 5 is a binarized image of an image captured by an IR camera in the present invention.

[0049] FIG. 5 shows the result of converting the original data captured by the IR camera (10) into grayscale and then binarizing it based on a specific threshold. The high-temperature area containing the flame is represented in white, and the remaining background area is represented in black. This binarization process is a process of clearly separating the high-temperature area and converting it into a preprocessing form for calculating the flame size.

[0050] That is, it is a preprocessing technique that determines whether there is a fire by calculating the ratio of the pixel size of the high-temperature region to the total pixel size of the IR camera image used and using the set ratio value. Since the IR camera (10) can detect temperature changes very sensitively, it has the characteristic of being able to recognize even small flames such as lighters or candles, but at the same time, there is a high possibility of causing problems such as false positives caused by small heat sources that are not actual fires.

[0051] To solve this problem, the fire monitoring system of the present invention proposes a method for determining the presence of a fire based on the absolute size of the high-temperature region or the proportion it occupies within the image. The IR camera image is converted to grayscale during the preprocessing stage and then binarized; in this process, the high-temperature region is represented as white (255) and the remaining region as black (0). The region represented as white in the binarized image is the part where the high-temperature flame exists, and the scale of the flame can be estimated by calculating the number of pixels occupied by this region in the entire image.

[0052] To this end, the size analysis unit (120) calculates the number of white (255) pixels in the image using the countNonZero function. The calculated number of white pixels is converted into a percentage relative to the total number of image pixels, and only if it is above a certain threshold value (e.g., 10%) is the high-temperature area and the rest are removed. Here, the threshold value can be set according to the resolution of the IR camera (10) used, so as to prevent false positives caused by small heat sources and to ensure that the fire is recognized only when the flame actually has a size greater than a certain amount.

[0053] Figure 6 shows the results of the analysis of the size ratio of the flame area among the total pixels in the present invention.

[0054] Figure 6 shows the results of measuring the size of the white area in the binarized image and illustrates the process of visually confirming how much the high-temperature area occupies within the image.

[0055] In the example of Fig. 6, the size of the white area is 3,578 pixels, which accounts for 18.64% of the total pixel size.

[0056] The shape analysis unit (130) detects the specific shape of the flame to increase the accuracy of flame detection and uses the Harris corner detection technique to easily extract corner points for shapes that are complex and irregular, such as flames.

[0057] The Harris corner detection technique is an algorithm that effectively extracts structural characteristics, characterized by its ability to easily identify corner points—locations within an image where pixel intensity changes abruptly. This offers the advantage of being well-suited to specific locations and shapes, such as flame boundaries or internal patterns. The Harris response function is calculated based on matrices and traces, and it determines corner status by utilizing information on intensity changes around each pixel. Through this process, it is possible to precisely distinguish whether a flame is merely a high-temperature region or possesses the actual structure of a flame.

[0058] Figure 7 illustrates the shape analysis results in the present invention.

[0059] Figure 7 shows the result of applying the Harris corner detection technique to an IR image, where the flame boundary and the irregular internal shape are visualized and emphasized as corner points. The detected corners clearly reveal the structural features constituting the shape of the flame region and are used to set the region of interest identified as a flame. That is, since the shape of a flame is characterized by having many corner points unlike ordinary objects, the principle is to estimate it as a flame if the number of corner points exceeding a predetermined set value occurs in a flame image detected at a high temperature of an appropriate size. Here, the set value of the corner points can be calculated based on the performance and resolution of the camera used. For example, the set value of the corner points can be calculated as 50 based on the experimental actual values ​​measured through the experiment of the present invention.

[0060] Flames do not remain in a fixed form but possess dynamic characteristics in which the center point continuously moves as time passes.

[0061] By utilizing these characteristics, the mobility analysis unit (140) analyzes the progress of the fire based on the movement pattern of the flame center.

[0062] For example, the mobility analysis unit (140) determines whether the flame is in a fixed state or moving by extracting flame center points (fire points or fire origins) at intervals of about 1 second and deriving a movement path by connecting each fire point with a line.

[0063] Figure 8 illustrates the analysis of flame movement over time in the present invention.

[0064] Figure 8 shows the results of visualizing the movement path of the center point immediately after the flame occurs. Initially, the center stays in a narrow area, but as time passes, it tends to move to a wider range.

[0065] Figure 9 visually illustrates the movement paths of flame center points in the present invention.

[0066] Figure 9 illustrates a result of visually highlighting the paths of flame center points extracted at 1-second intervals, and such analysis can be used to quantitatively evaluate the direction of flame spread and the speed of progression.

[0067] In the example of Fig. 9, it can be seen that 3 pixels are moved between 1 second and 2 seconds, and 7 pixels are moved between 2 seconds and 3 seconds.

[0068] In the fire detection system of the present invention, the movement area of ​​the flame is quantitatively evaluated based on the movement path of the flame's center, and this is applied as the final step of the preprocessing stage of a deep learning model such as YOLO v8. The movement range of the flame's center and the location information of its occurrence are comprehensively analyzed to define a Fire Region of Interest (F-ROI), and based on this, it is determined whether it is an actual flame. This preprocessing process serves as an important foundation for reducing unnecessary false positives in actual fire situations and increasing the accuracy and reliability of flame detection in deep learning models including the YOLO v8 algorithm.

[0069] In the experiments of the present invention, a flame detection system was designed based on the YOLO v8 Nano model. YOLO v8 can effectively detect flame objects with complex boundaries and various sizes through SPPF and PANet structures, and the Nano model is suitable for real-time processing due to its lightweight structure. Prior to training, IR camera images undergo preprocessing steps such as grayscale conversion, binarization, contour detection, and center point tracking to highlight the characteristics and location of the flames. This defines the Fire Region of Interest (F-ROI) and improves the efficiency of model training and detection accuracy. Through the experiments of the present invention, it can be confirmed that the system proposed in this invention exhibits high performance and reliability even in various environments.

[0070] Figure 10 illustrates the flame detection analysis results using deep learning in the present invention, showing the results of flame detection using the YOLO v8 model.

[0071] In this invention, a process is performed to precisely derive fire interest region candidates through four image preprocessing techniques regarding temperature, size, shape, and movement, and this method of configuring training data based on such preprocessing plays a decisive role in improving the accuracy and precision of flame detection.

[0072] Figure 11 is a graph showing the results of fire detection analysis using deep learning in an experiment of the present invention.

[0073] Referring to Figure 11, the actual analysis results from the experiment of the present invention showed that the YOLO v8 Nano model demonstrated excellent performance, recording an average accuracy of 96.5%, a precision of 95%, and a recall of 97.9% on the test dataset. This is because the deep learning model was configured to focus on learning the actual flame area through a preprocessing process. In particular, the YOLO v8 Nano model demonstrated that real-time high-performance flame detection is possible even in a limited resource environment based on a lightweight architecture.

[0074] Although the present invention has been described above using several preferred embodiments, these embodiments are illustrative and not limiting. Those skilled in the art will understand that various changes and modifications can be made without departing from the spirit of the invention and the scope of rights set forth in the appended claims. Explanation of the symbols

[0076] 100 Image Preprocessing Unit 200 Deep Learning Unit 110 High-temperature region extraction unit 120 Size analysis unit 130 Shape Analysis Unit 140 Mobility Analysis Unit

Claims

Claim 1 A fire detection system characterized by comprising: an image preprocessing unit that preprocesses an IR image, which is an image captured by an IR camera, to define a fire interest region; and a deep learning unit that determines whether a fire exists based on a deep learning model that takes the fire interest region defined in the image preprocessing unit as input, wherein the image preprocessing unit comprises: a high-temperature region extraction unit that extracts a high-temperature region, which is a region above a reference temperature in the IR image; a size analysis unit that removes the remaining high-temperature regions from the high-temperature regions extracted by the high-temperature region extraction unit, leaving only the high-temperature regions that are above a predetermined reference size; a shape analysis unit that recognizes the flame shape by analyzing the outer shape and internal pattern of the remaining high-temperature regions through the size analysis unit; and a mobility analysis unit that analyzes the movement path of the flame recognized by the shape analysis unit to distinguish between a static heat source and a dynamic flame, and defines the dynamic flame region as a fire interest region. Claim 2 delete Claim 3 A fire detection system according to claim 1, wherein the high-temperature region extraction unit performs a binarization process in which the high-temperature region is converted to white and the remaining region is converted to black, and extracts the high-temperature region by extracting the outer contour of the high-temperature region. Claim 4 A fire detection system according to claim 1, wherein the shape analysis unit analyzes the shape by detecting corner points of a high-temperature region using a Harris corner detection technique. Claim 5 A fire detection system according to claim 4, wherein the shape analysis unit recognizes a flame shape when the number of corner points is greater than or equal to a predetermined threshold. Claim 6 A fire detection system according to claim 1, wherein the deep learning unit determines whether there is a fire based on the YOLO v8 Nano model.

Images