Multimodal Fire Detection System

Abstract

A multimodal fire detection system according to one embodiment of the present invention fuses data collected from physical sensors including an IR sensor, a thermal imaging sensor, and a smoke sensor to calculate a fire score, analyzes image data from an RGB camera to calculate a non-fire suppression score, and stabilizes time-series data of a fire discrimination score calculated by reflecting the non-fire suppression score in the fire score using an RNN or GRU-based time-series model, thereby configuring a system that suppresses false alarms caused by instantaneous sensor reactions. Through this, false alarms caused by non-fire situations such as welding, cutting, and smoking in industrial sites are effectively suppressed, and the reliability of fire detection is improved.

Description

Technology Field

[0001] The present invention discloses technology related to a fire detection system. Background Technology

[0002] Fire detection systems serve as a core safety infrastructure for minimizing casualties and property damage, and are widely applied in various environments, including industrial sites, public facilities, and residential areas.

[0003] Conventional fire detection systems typically relied on a single sensor, such as a smoke sensor or a heat detector. However, single-sensor-based fire detection systems suffered from a serious problem of false alarms, which triggered alarms even in non-fire situations that frequently occur in industrial settings, such as welding, cutting, cooking, and smoking.

[0004] Such false alarms undermine system reliability and dull user vigilance, causing problems that hinder a rapid response in the event of an actual fire.

[0005] To address this, research has been conducted on multimodal fire detection systems that fuse multiple sensors (e.g., composite fire detection systems combining image data and physical sensor data).

[0006] However, the above conventional technologies have the following problems.

[0007] First, most conventional multi-sensor fusion systems are limited to simply combining image data and physical sensor data, and have the limitation of not being able to utilize operational noise, such as welding and cutting sounds—key non-fire factors in industrial sites—as a discrimination element.

[0008] Second, conventional systems often detect fires by simply comparing data collected from multiple sensors with threshold values, which has the problem of failing to effectively suppress false alarms caused by instantaneous sensor reactions.

[0009] Third, conventional AI-based fire detection systems often rely on transmitting data to cloud servers to perform inference; consequently, fire detection becomes impossible in the event of communication failures, and there are concerns regarding privacy infringement due to the continuous transmission of video data.

[0010] Fourth, conventional time-series analysis-based fire detection systems use a method of directly inputting raw sensor data into a time-series model, and thus fail to disclose a structure that stabilizes the result of fusing multiple sensor data into a time-series model.

[0011] The present invention was devised to solve the problems of the prior art as described above. The problem to be solved

[0012] The present invention was devised to solve the problems of the prior art as described above, and aims to solve the following problems.

[0013] First, the purpose is to provide a multimodal sensor fusion-based fire detection system that calculates a fire discrimination score by reflecting the non-fire suppression score in the fire score.

[0014] Second, the present invention aims to provide a system that precisely identifies non-fire situations frequently occurring in industrial sites and suppresses false alarms by classifying non-fire acoustic patterns including welding sounds, cutting sounds, and environmental noise.

[0015] Third, the present invention aims to provide a system that suppresses false alarms caused by instantaneous sensor responses and outputs stabilized fire detection results by performing time-series pattern analysis.

[0016] Fourth, the present invention aims to provide a system capable of accurate fire detection while optimizing computational efficiency. means of solving the problem

[0017] A multimodal fire detection system according to one embodiment of the present invention for solving the above-mentioned problem comprises: a system that calculates a fire score by fusing data collected from physical sensors including an IR sensor, a thermal imaging sensor, and a smoke sensor; calculates a non-fire suppression score by analyzing image data from an RGB camera; and stabilizes the time series data of the fire discrimination score calculated by reflecting the non-fire suppression score in the fire score using a time series model based on an RNN, GRU, LSTM, or AutoEncoder to suppress false alarms caused by instantaneous sensor responses.

[0018] A multimodal fire detection system according to another embodiment of the present invention comprises a conditional sequential structure that determines whether to execute a non-fire suppression score calculation operation based on the magnitude of the fire score. Effects of the invention

[0019] The multimodal fire detection system according to the present invention provides the following effects.

[0020] First, the present invention has the effect of improving fire detection accuracy compared to a single-sensor-based system and significantly reducing the false alarm rate.

[0021] Second, the present invention has the effect of precisely identifying non-fire situations in industrial sites that are difficult to distinguish based solely on image data. In particular, it can effectively suppress false alarms caused by non-fire situations that frequently involve noise in industrial sites, such as welding and cutting operations.

[0022] Third, the present invention has the effect of suppressing false alarms that misidentify a temporary score increase caused by instantaneous sensor response as a fire, and selectively outputting only continuous fire patterns as an alarm. In particular, by using fused fire detection scores, rather than raw sensor data, as input to a time-series model, it has the effect of improving the accuracy of time-series stabilization compared to conventional technology.

[0023] Fourth, the present invention enables immediate alarm response by omitting the non-fire suppression score calculation operation in sections with a high fire risk, and optimizes computational efficiency in an on-device environment by not performing unnecessary calculations in sections with a low fire risk.

[0024] Fifth, the present invention has the effect of preventing the continuous transmission of video data during normal operation, thereby enhancing the protection of personal information and industrial secrets and minimizing network bandwidth usage.

[0025] However, the effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0026] FIG. 1 is a block diagram showing the hardware configuration of a multimodal fire detection system according to one embodiment of the present invention. FIG. 2 is a flowchart showing the overall operation flow of a multimodal fire detection system according to one embodiment of the present invention. FIG. 3 is a flowchart showing the detailed flow of a physical sensor-based fire score calculation operation according to one embodiment of the present invention. FIG. 4 is a flowchart showing the detailed flow of a non-fire suppression score calculation operation according to one embodiment of the present invention. FIG. 5 is a flowchart showing the detailed flow of an acoustic non-fire suppression score calculation operation according to one embodiment of the present invention. FIG. 6 is a flowchart showing the branching flow of a fire detection operation according to a conditional sequential structure according to an embodiment of the present invention. Specific details for implementing the invention

[0027] The foregoing and additional aspects are embodied through embodiments described with reference to the attached drawings. It is understood that the components of each embodiment may be combined in various ways within the embodiment or with components of other embodiments, unless otherwise stated or contradicted. Based on the principle that the inventor may appropriately define the concepts of terms to best describe his invention, the terms used in this specification and claims shall be interpreted in a meaning and concept consistent with the description or proposed technical idea.

[0028] Blocks referred to as 'circuits' or 'parts' with reference to block diagrams in this specification may be designed as circuits including transistors, inductors, and capacitors, or may be composed of hardware or parts thereof, such as semiconductor design assets (IP), gate arrays, and FPGAs. One or more blocks may be implemented as a single piece of hardware. As another example, these blocks may be implemented in software as an information processing device in which a computation element executes program instructions stored in a memory element. Multiple blocks may be implemented as part of a program executed on the same computation element. As yet another example, these blocks may be implemented in a hybrid form in which part of the individual circuit is hardware and part is software. Furthermore, in a software implementation, the computation element may include digital signal processors, dedicated computation processors, artificial intelligence processing engines, dedicated artificial intelligence processors, and graphics processors, or a combination thereof to the extent possible.

[0029] In this document, program instruction(s) may mean 'executable instructions'. Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings.

[0030] FIG. 1 is a block diagram showing the hardware configuration of a multimodal fire detection system (1000) according to an embodiment of the present invention. Referring to FIG. 1, a multimodal fire detection system (1000) according to an embodiment of the present invention includes a memory (100), a processor (200), and a communication circuit (300).

[0031] Memory (100) stores program instructions for performing operations of the present invention. Memory (100) may include at least one of volatile memory (e.g., DRAM, SRAM) and non-volatile memory (e.g., Flash, eMMC). Memory (100) stores program instructions for performing physical sensor-based fire score calculation operations (O100), non-fire suppression score calculation operations (O200), fire detection score calculation operations (O300), fire detection score stabilization operations (O400), and fire alarm issuance operations (O500). Additionally, memory (100) may store parameters of a neural network model used for physical sensor-based fire score calculation, a neural network model used for acoustic feature classification, and an RNN or GRU-based time series model used for fire detection score stabilization.

[0032] The processor (200) executes program instructions stored in memory (100). The processor (200) may include at least one of a CPU (Central Processing Unit), an NPU (Neural Processing Unit), and a GPU (Graphics Processing Unit). In one embodiment, the processor (200) may be a processor mounted on an edge device, such as an NVIDIA Jetson Orin or a Raspberry Pi and HAIL NPU, or an MCU NPU combined CPU such as an STM32N6, but is not limited thereto. According to one embodiment, the processor (200) processes all inference processes in an on-device environment without communication with an external server, thereby enabling real-time fire detection even in the event of a communication failure.

[0033] The communication circuit (300) connects communication between the multimodal fire detection system (1000) and an external device. The communication circuit (300) may support wired or wireless communication methods and may support at least one of communication protocols such as Wi-Fi, Ethernet, Bluetooth, and LTE. The communication circuit (300) performs the function of transmitting the fire alarm issuance result to an alarm device, or selectively transmitting video data to an external server and time-series data to a data server when an event occurs. A plurality of sensors are connected to the multimodal fire detection system (1000). Specifically, a physical sensor (2000) including an IR sensor (2100), a thermal imaging sensor (2200), and a smoke sensor (2300), an RGB camera (3000), and a microphone (4000) are connected to the processor (200) to transmit each sensor data to the processor (200).

[0034] The IR sensor (2100) measures the point temperature of the detection target area and outputs temperature data. The IR sensor (2100) can be implemented as a non-contact temperature sensor capable of measuring the surface temperature of an object without contact.

[0035] The thermal imaging sensor (2200) outputs the heat distribution of the detection target area in the form of an image. The thermal imaging sensor (2200) detects infrared radiation energy emitted from an object and generates a heat pattern image. The resolution of the thermal imaging sensor (2200) may be 32×24 pixels or higher, but is not limited thereto.

[0036] The smoke sensor (2300) measures the concentration of particles in the air of the detection target area and outputs smoke concentration data. The smoke sensor (2300) can be implemented using at least one of a photoelectric or ionization smoke detection method.

[0037] The RGB camera (3000) collects visual information of the detection target area. The RGB camera (3000) outputs video data in real time that is necessary for calculating a visual non-fire suppression score, such as flame patterns, smoke shapes, and worker behavior.

[0038] The microphone (4000) collects acoustic data of the detection target area. The microphone (4000) outputs acoustic data in real time that is necessary for classifying non-fire acoustic patterns, such as welding sounds, cutting sounds, and environmental noise.

[0039] FIG. 2 is a flowchart showing the overall operation flow of a multimodal fire detection system (1000) according to one embodiment of the present invention. Referring to FIG. 2, according to one embodiment, the multimodal fire detection system (1000) may sequentially execute a physical sensor-based fire score calculation operation (O100), a non-fire suppression score calculation operation (O200), a fire discrimination score calculation operation (O300), a fire discrimination score stabilization operation (O400), and a fire alarm issuance operation (O500), or the physical sensor-based fire score calculation operation (O100) and the non-fire suppression score calculation operation (O200) may be executed simultaneously.

[0040] In the physical sensor-based fire score calculation operation (O100), the processor (200) fuses physical sensor data collected from the physical sensor (2000) to calculate a fire score (Fire_Score). The fire score (Fire_Score) can be normalized to a value between 0 and 1, and a higher value may indicate a higher probability of fire. The detailed flow of the physical sensor-based fire score calculation operation (O100) will be described later with reference to FIG. 3.

[0041] In the non-fire suppression score calculation operation (O200), the processor (200) analyzes image data collected from the RGB camera (3000) to calculate the non-fire suppression score (NonFire_Suppress).

[0042] The NonFire_Suppress score is a value between 0 and 1, and a higher value indicates a higher probability that the current situation is a nonfire situation. The detailed flow of the NonFire_Suppress score calculation operation (O200) will be described later with reference to FIG. 4.

[0043] In the fire detection score calculation operation (O300), the processor (200) can calculate the fire detection score (Final_Score) by reflecting the non-fire suppression score (NonFire_Suppress) in the fire score (Fire_Score). In one embodiment, the fire detection score (Final_Score) can be calculated according to the following mathematical formula 1.

[0044] - Mathematical formula 1

[0045] Final_Score = Fire_Score × (1 - NonFire_Suppress)

[0046] According to mathematical formula 1, even if the Fire_Score is high, the Fire_Score may be lowered if the NonFire_Suppress score is high.

[0047] For example, even if the Fire_Score is calculated to be high at 0.8 due to welding work, if the welding sound and welding flame pattern are detected and the NonFire_Suppress score is calculated to be 0.85, the Final_Score is calculated to be low at 0.8 × (1 - 0.85) = 0.12, thereby suppressing false alarms.

[0048] In the fire detection score stabilization operation (O400), the processor (200) collects time series data of fire detection scores (Final_Score) over a recent predetermined period in a sliding window manner and inputs them into a time series model based on RNN, GRU, LSTM, or AutoEncoder to perform time series pattern analysis. Through this, false alarms caused by instantaneous sensor responses are suppressed, and a stabilized fire detection score (Final_Score_stable) is output.

[0049] In one embodiment, the predetermined period may be approximately 5 seconds, but is not limited thereto and may be adjusted according to the installation environment. The sliding window operates by removing the oldest sample and adding the newest sample at each inference cycle. For example, if the sampling cycle is 0.5 seconds, the sliding window contains up to 10 fire detection score samples.

[0050] A time series model based on RNN, GRU, LSTM, or AutoEncoder takes fire detection score time series data within a sliding window as input, distinguishes whether the time series is a transient pattern caused by instantaneous sensor response or a continuous pattern caused by actual fire, and outputs a stabilized fire detection score (Final_Score_stable).

[0051] In one embodiment, a GRU (Gated Recurrent Unit)-based time series model may be used. GRU is suitable for real-time inference in an on-device environment because it has fewer parameters than LSTM (Long Short-Term Memory).

[0052] In the fire alarm issuance operation (O500), the processor (200) determines whether to issue a fire alarm by comparing the stabilized fire detection score (Final_Score_stable) with a predetermined alarm threshold. In one embodiment, the alarm threshold may be 0.75, but is not limited thereto and may be adjusted according to the installation environment.

[0053] When the stabilized fire detection score (Final_Score_stable) is greater than or equal to the alarm threshold, the processor (200) issues a fire alarm through the communication circuit (300).

[0054] In one embodiment, when a fire alarm is issued, the processor (200) may perform a result notification transmission operation (O910) that immediately transmits the result of the fire alarm issuance to an alarm device (e.g., speaker, LED, alarm), an image data transmission operation (O920) that selectively transmits image data collected from an RGB camera (3000) to an external server (not shown) only when an event occurs, and a time series data transmission operation (O930) that transmits time series data of the fire detection score to a data server only when an event occurs.

[0055] As such, since video data and time-series data are selectively transmitted only when a fire event occurs, the continuous transmission of video data during normal times is prevented, thereby enhancing personal information protection and minimizing network bandwidth usage.

[0056] FIG. 3 is a flowchart showing the detailed flow of a physical sensor-based fire score calculation operation (O100) according to an embodiment of the present invention. Referring to FIG. 3, the physical sensor-based fire score calculation operation (O100) may include an IR score calculation operation (O110), a thermal image score calculation operation (O120), a smoke score calculation operation (O130), and a weighted sum calculation operation (O140).

[0057] In the IR score calculation operation (O110), the processor (200) normalizes the temperature data measured from the IR sensor (2100) to calculate the IR score (IR_Score).

[0058] In one embodiment, the IR score (IR_Score) can be calculated as a value between 0 and 1 by linearly normalizing the measured temperature over a predetermined minimum and maximum temperature range. For example, if the measured temperature is less than 50°C, the IR score can be calculated as a value close to 0, and if it is 200°C or higher, the IR score can be calculated as a value close to 1. The temperature range can be set as an adjustable parameter according to the characteristics of the installation environment.

[0059] In the thermal image score calculation operation (O120), the processor (200) can calculate a thermal image score (Thermal_Score) by analyzing a thermal pattern image acquired from a thermal image sensor (2200) using a neural network model. The neural network model receives the thermal pattern image as input and classifies whether it is a fire thermal pattern.

[0060] In one embodiment, the neural network model may be implemented as a lightweight Convolutional Neural Network (CNN) model such as MobileNetV3, but is not limited thereto.

[0061] The neural network model can distinguish and classify normal heat distribution patterns, local high temperature patterns, diffuse high temperature patterns, etc., and a Thermal_Score can be calculated based on the classification results. The Thermal_Score can be calculated as a value between 0 and 1.

[0062] In the smoke score calculation operation (O130), the processor (200) can calculate a smoke score (Smoke_Score) based on the particle concentration and concentration change rate measured from the smoke sensor (2300).

[0063] In one embodiment, the Smoke_Score can be calculated by combining a concentration score normalized from the current particle concentration and a change rate weighting that reflects the rate of change in particle concentration per unit time (Δppm / s). When the rate of change in concentration is high (e.g., rapid smoke generation), a high change rate weighting is applied to increase the Smoke_Score, and when the rate of change in concentration is low (e.g., cigarette smoke, cooking smoke), a low change rate weighting is applied to suppress the Smoke_Score.

[0064] Through this, slowly increasing non-fire smoke and rapidly spreading fire smoke can be effectively distinguished. The Smoke_Score is calculated as a value between 0 and 1. In the weighted sum calculation operation (O140), the processor (200) can calculate the Fire_Score as a weighted sum by applying a predetermined weight to each of the IR_Score, Thermal_Score, and Smoke_Score.

[0065] In one embodiment, the Fire_Score can be calculated according to the following mathematical formula 2.

[0066] - Mathematical formula 2

[0067] Fire_Score = α × IR_Score + β × Thermal_Score +γ × Smoke_Score

[0068] (However, α + β + γ = 1, 0 < α, β, γ < 1)

[0069] For example, α = 0.3, β = 0.4, and γ = 0.3 can be set. A relatively high weight (β = 0.4) is applied to the Thermal_Score because the thermal imaging sensor (2200) contains spatial heat distribution pattern information, which has a higher reliability in fire detection compared to the IR sensor (2100) and the smoke sensor (2300).

[0070] However, the above weight values ​​are exemplary and can be set as adjustable parameters depending on the installation environment profile (e.g., welding plant, office, kitchen).

[0071] FIG. 4 is a flowchart showing the detailed flow of a non-fire suppression score calculation operation (O200) according to an embodiment of the present invention. Referring to FIG. 4, the non-fire suppression score calculation operation (O200) may include a visual non-fire suppression score calculation operation (O210), an acoustic non-fire suppression score calculation operation (O220), and a non-fire suppression score combination calculation operation (O230).

[0072] In the visual non-fire suppression score calculation operation (O210), the processor (200) can calculate the visual non-fire suppression score (Visual_Score) by analyzing image data collected from the RGB camera (3000).

[0073] In one embodiment, the processor (200) can detect flame patterns, smoke shapes, worker behaviors, etc. from image data using an object detection model such as YOLOv8. A visual non-fire suppression score (Visual_Score) can be calculated based on the type and reliability of the detected objects.

[0074] For example, if welding flame patterns, cutting spark patterns, or worker welding behavior are detected, the Visual Non-Fire Score (Visual_Score) may be calculated as high, and if flame spread patterns or smoke spread patterns are detected, the Visual Non-Fire Score (Visual_Score) may be calculated as low.

[0075] The Visual_Score can be calculated as a value between 0 and 1.

[0076] In the acoustic non-fire suppression score calculation operation (O220), the processor (200) can calculate an acoustic-based non-fire suppression score (Audio_Score) by applying Mel-Frequency Cepstral Coefficients (MFCC) feature extraction to acoustic data collected from the microphone (4000) and classifying non-fire acoustic patterns through a neural network model. The detailed flow of the acoustic non-fire suppression score calculation operation (O220) will be described later with reference to FIG. 5.

[0077] In the nonfire suppression score combination calculation operation (O230), the processor (200) can calculate a nonfire suppression score (NonFire_Suppress) by combining the visual nonfire suppression score (Visual_Score) and the audio-based nonfire suppression score (Audio_Score).

[0078] In one embodiment, the NonFire_Suppress score is calculated according to the following mathematical formula 3.

[0079] - Mathematical formula 3

[0080] NonFire_Suppress = δ × Visual_Score +ε × Audio_Score

[0081] (However, δ + ε = 1, 0 < δ, ε < 1)

[0082] For example, δ = 0.6 and ε = 0.4 can be set. A relatively high weight (δ = 0.6) is applied to the visual non-fire suppression score (Visual_Score) because image-based object detection is more robust to environmental interference compared to acoustic analysis.

[0083] The acoustic-based non-fire suppression score (Audio_Score) plays a complementary role in identifying non-fire operational noise occurring in the camera's blind spots. However, the above weighting values ​​are exemplary and can be set as parameters adjustable according to the installation environment.

[0084] FIG. 5 is a flowchart showing the detailed flow of an acoustic non-fire suppression score calculation operation (O220) according to an embodiment of the present invention. Referring to FIG. 5, the acoustic non-fire suppression score calculation operation (O220) includes an acoustic feature vector generation operation (O221), a non-fire acoustic pattern classification operation (O222), and a non-fire suppression score determination operation (O223).

[0085] In the acoustic feature vector generation operation (O221), the processor (200) generates an acoustic feature vector by applying Mel-Frequency Cepstral Coefficients (MFCC) feature extraction to acoustic data collected from the microphone (4000).

[0086] MFCC is a feature extraction method that converts the spectral characteristics of an acoustic signal into a Mel scale that mimics human hearing characteristics, and can effectively represent the frequency characteristics of industrial noise. For example, the number of MFCC coefficients can be 13 or more, and the frame length and hop size of the acoustic data can be adjusted according to the installation environment.

[0087] In the non-fire acoustic pattern classification operation (O222), the processor (200) inputs an acoustic feature vector into a neural network model and classifies it into a non-fire acoustic pattern including at least one of welding sound, cutting sound and environmental noise.

[0088] In one embodiment, the neural network model may be implemented as a CNN+LSTM model combining a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM), but is not limited thereto.

[0089] The frequency characteristics of non-fire acoustic patterns are as follows. Welding arc sound is dominated by frequency components in the 400Hz to 800Hz band, and cutting friction sound is dominated by high-frequency components in the 1kHz to 8kHz band. Ambient noise has low-energy broadband frequency components. Fire crackling sound (burning sound) has irregular transient components along with a broadband energy distribution and is distinguished from the above non-fire acoustic patterns.

[0090] The neural network model learns these frequency characteristics to classify the input acoustic feature vector into non-fire acoustic patterns or fire-related acoustic patterns, and outputs a classification probability value for each class.

[0091] In the non-fire suppression score determination operation (O223), the processor (200) determines an acoustic-based non-fire suppression score (Audio_Score) based on the classification result of the non-fire acoustic pattern classification operation (O222).

[0092] In one embodiment, the classification probability value for a non-fire acoustic pattern output by a neural network model can be used as an acoustic-based non-fire suppression score (Audio_Score).

[0093] For example, if the probability of being classified as a welding arc sound is 0.85, the acoustic-based non-fire suppression score (Audio_Score) is determined to be 0.85. The acoustic-based non-fire suppression score (Audio_Score) can be determined to a value between 0 and 1.

[0094] FIG. 6 is a flowchart showing the branching flow of a fire determination operation according to a conditional sequential structure according to an embodiment of the present invention. Referring to FIG. 6, in an embodiment of the present invention, a processor (200) can perform a conditional sequential structure that determines whether to execute a subsequent operation based on the magnitude of the calculated fire score (Fire_Score) after a physical sensor-based fire score calculation operation (O100).

[0095] For example, if the fire score (Fire_Score) is greater than or equal to the first threshold (e.g., Fire_Score ≥ 0.7), the processor (200) executes a fire detection score stabilization entry operation (O600) that skips the non-fire suppression score calculation operation (O200) and enters the fire detection score stabilization operation (O400) by considering the fire score (Fire_Score) as the fire detection score. This is intended to enable immediate alarm response by skipping the computation required for calculating the non-fire suppression score in the high-risk section where the fire score (Fire_Score) is greater than or equal to the first threshold. In this case, the fire detection score is set to the same value as the fire score (Fire_Score) and is used as the input to the fire detection score stabilization operation (O400).

[0096] When the fire score (Fire_Score) is less than the second threshold value (e.g., Fire_Score < 0.3), the processor (200) determines that the state is non-fire and executes a fire alarm non-issuance operation (O700) that does not issue a fire alarm. This is to optimize computational efficiency in an on-device environment by not performing unnecessary operations for calculating the non-fire suppression score and stabilizing the fire detection score during normal periods when the fire score (Fire_Score) is sufficiently low.

[0097] When the Fire_Score is greater than or equal to the second threshold and less than the first threshold (e.g., 0.3 ≤ Fire_Score < 0.7), the processor (200) determines that a section requiring non-fire verification is a section and can execute a non-fire suppression score calculation entry operation (O800) to enter a non-fire suppression score calculation operation (O200). In this section, since there is a possibility that non-fire situations at industrial sites, such as welding, cutting, and smoking, and the initial stages of actual fires may be mixed, a secondary verification is performed through the non-fire suppression score calculation operation (O200) to suppress false alarms. In one embodiment, the first threshold may be set to 0.7 and the second threshold to 0.3, but is not limited thereto and may be set as parameters adjustable according to the installation environment. The first threshold and the second threshold may be manually adjusted in the administrator setting mode or automatically selected according to the installation environment profile.

[0098] The conditional sequential structure described above is a specific embodiment of the basic sequential structure (O100 → O200 → O300 → O400 → O500) described in FIG. 2, and optimizes computational efficiency by determining whether to execute the non-fire suppression score calculation operation (O200) based on the size of the fire score (Fire_Score).

[0099] Hereinafter, a specific embodiment of the multimodal fire detection system (1000) of the present invention determining major non-fire scenarios is described.

[0100] [Example 1: Welding Work Environment]

[0101] In a welding work environment, radial flames, localized high temperatures, and welding arc sounds (400Hz to 800Hz) are generated. In this case, in the physical sensor-based fire score calculation operation (O100), the thermal imaging sensor (2200) detects a localized high temperature pattern and the IR sensor (2100) measures a high temperature, so the fire score (Fire_Score) may increase. However, in the non-fire suppression score calculation operation (O200), the RGB camera (3000) detects a radial welding flame pattern and the microphone (4000) detects a welding arc sound (400Hz to 800Hz), so the non-fire suppression score (NonFire_Suppress) is calculated to be high. According to Equation 1, the high non-fire suppression score is reflected in the fire score, so the fire determination score (Final_Score) is calculated to be low, thereby suppressing false alarms.

[0102] [Example 2: Cutting Operation Environment]

[0103] In a cutting operation environment, metal sparks, single high temperatures, and high-frequency friction sounds (1 kHz to 8 kHz) are generated. A microphone (4000) detects the high-frequency components of the cutting friction sound, and a neural network model classifies them as non-fire acoustic patterns, thereby generating a high acoustic-based non-fire suppression score (Audio_Score) and suppressing false alarms.

[0104] [Example 3: Actual Fire Occurrence]

[0105] When an actual fire occurs, the spread of flames and smoke, sustained high temperatures, and crackling sounds (burning sounds) occur. In this case, the Fire_Score is consistently calculated as high, and the NonFire_Suppress score is calculated as low, resulting in a high Final_Score. Additionally, during the Fire_Score stabilization operation (O400), a time series model based on RNN, GRU, LSTM, or AutoEncoder recognizes the continuous rising pattern of the Fire_Score as a fire pattern, and the stabilized Fire_Score (Final_Score_stable) is calculated to be above the alarm threshold, thereby triggering a fire alarm.

[0106] Although the present invention has been described above with reference to embodiments with reference to the accompanying drawings, it is not limited thereto and should be interpreted to encompass various variations that can be obviously derived from them by those skilled in the art. The claims are intended to encompass such variations. Explanation of the symbols

[0107] 100: Memory 200: Processor 300: Communication circuit 1000: Multimodal Fire Detection System 2000: Physical sensor 2100: IR sensor 2200: Thermal imaging sensor 2300: Smoke sensor 3000: RGB camera 4000: Microphone

Claims

Claim 1 Memory storing program instructions; at least one processor executing the program instructions; A multimodal fire detection system comprising: a communication circuit connecting communication with an external device; wherein, when the program instructions are executed by the processor, the system comprises: a physical sensor-based fire score calculation operation that calculates a fire score by fusing physical sensor data collected from a physical sensor including at least one of an IR sensor, a thermal imaging sensor, and a smoke sensor; a visual non-fire suppression score calculation operation that calculates a visual non-fire suppression score by analyzing image data collected from an RGB camera; an acoustic non-fire suppression score calculation operation that classifies acoustic feature vectors generated by applying Mel-Frequency Cepstral Coefficients (MFCC) feature extraction to acoustic data collected from a microphone into a neural network model to classify them into non-fire acoustic patterns including at least one of welding sound, cutting sound, and environmental noise, and calculates an acoustic-based non-fire suppression score based on the classification result; and a non-fire suppression score combination calculation operation that calculates a non-fire suppression score by combining the visual non-fire suppression score and the acoustic-based non-fire suppression score; and fire determination by multiplying the fire score by (1 - the non-fire suppression score). A fire detection score calculation operation for calculating a score; a fire detection score stabilization operation for collecting time series data of the fire detection score over a recent predetermined period using a sliding window method, inputting it into a time series model based on RNN, GRU, LSTM, or AutoEncoder to suppress false alarms caused by instantaneous sensor responses through time series pattern analysis, and outputting a stabilized fire detection score; and a fire alarm issuance operation for issuing a fire alarm when the stabilized fire detection score is greater than or equal to a predetermined alarm threshold.A multimodal fire detection system configured to execute: a fire discrimination score stabilization entry operation in which, when the fire score is greater than or equal to a first threshold, the non-fire suppression score calculation operation is omitted and the fire score is considered as a fire discrimination score to enter the fire discrimination score stabilization operation; a fire alarm non-issuance operation in which, when the fire score is less than a second threshold, a non-fire state is determined and a fire alarm is not issued; and a non-fire suppression score calculation entry operation in which, when the fire score is greater than or equal to the second threshold and less than the first threshold, the system enters the non-fire suppression score calculation operation. Claim 2 A multimodal fire detection system according to claim 1, wherein the physical sensor-based fire score calculation operation comprises: an IR score calculation operation that calculates an IR score by normalizing temperature data measured from the IR sensor; a thermal image score calculation operation that calculates a thermal image score by analyzing a thermal pattern image acquired from the thermal image sensor using a neural network model; a smoke score calculation operation that calculates a smoke score based on particle concentration and concentration change rate measured from the smoke sensor; and a weighted sum calculation operation that calculates the fire score as a weighted sum by applying a predetermined weight to each of the IR score, the thermal image score, and the smoke score. Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete

Images