AI-based Multi-spectral Image Fusion Fire Monitoring System

Abstract

The present invention relates to a system for predicting the location of a fire and the direction of its spread in real time by simultaneously collecting visible light images, infrared thermal images, and smoke diffusion pattern images to detect flames, smoke, and heat sources at an early stage, and by fusion analyzing the collected multispectral data using an artificial intelligence (AI) algorithm. The effects of the present invention are as follows. First, through multi-sensor data fusion, it provides robust sensing performance against weather conditions or changes in the surrounding environment. Second, through the analysis of the flame's fractal contours and infrared temperature rise patterns, it is possible to clearly distinguish phenomena similar to actual fires. Third, the organic connection between the field control unit and the central control server enables integrated fire management across a wide area.

Description

Technology Field

[0001] The present invention relates to a system for predicting the location of a fire and the direction of its spread in real time by simultaneously collecting visible light images, infrared thermal images, and smoke diffusion pattern images to detect flames, smoke, and heat sources at an early stage, and by fusion analyzing the collected multispectral data using an artificial intelligence (AI) algorithm. Background Technology

[0002] Conventional fire monitoring systems have primarily relied on visible light-based CCTV video analysis or simple smoke and heat detection sensors to detect fires. This approach has fundamental limitations in securing the golden time for initial suppression because it only activates when the fire has already progressed beyond a certain level, making flames visible to the naked eye or when smoke concentration exceeds a critical threshold. Furthermore, in adverse external environments such as night, fog, or heavy snow, the accuracy of image recognition deteriorates rapidly, which can lead to false positives or missed detections.

[0003] Recently, there have been ongoing attempts to improve the accuracy of fire detection by utilizing artificial intelligence (AI) and multispectral imaging fusion technology to superimpose visible light and infrared thermal images for analysis. However, even these technologies remain limited to passive detection methods that track "generated signals" that are outwardly visible, such as flames or smoke. In particular, active monitoring systems capable of preventing fires by intelligently capturing subtle physical changes or atypical warning signs in the surrounding environment just before ignition are still lacking.

[0004] Fire monitoring systems are increasingly being emphasized as asset protection tools directly linked to corporate risk management, going beyond the scope of mere security equipment. Nevertheless, there are structural limitations where diligent system operation by managers or long-term accident-free records do not translate into real-time financial benefits, such as actual insurance premium calculations. Therefore, it is time for an evolution into an intelligent risk management platform that combines precise AI analysis data with accident-free history to calculate insurance rates in real time and provide economic incentives to managers. Prior art literature

[0005] Korean Registered Patent Publication No. 10-2499342 (February 6, 2023) The problem to be solved

[0006] The objective of the present invention is as follows.

[0007] First, it is intended to resolve the malfunction problems caused by light reflection, water vapor, shadows, etc., that frequently occur in existing single-image-based systems.

[0008] Second, it is intended to detect signs of fire in advance by analyzing smoke particle concentration and thermal radiation energy in a time series.

[0009] Third, it is intended to maximize the efficiency of initial suppression by immediately identifying the location of the fire and the direction of its spread. means of solving the problem

[0010] To solve the above problems, the present invention includes a composite camera unit (100) installed in a fire monitoring target area to collect multispectral data related to fire in real time, and a fire integrated control server (200) for determining whether a fire has occurred by analyzing data collected from the composite camera unit (100). The composite camera unit (100) includes a visible light image sensor (110) for capturing images corresponding to the human visible light range to obtain visual information of the scene, an infrared sensor (120) for collecting temperature information by detecting thermal radiation energy emitted from an object, and a scattered light detection sensor (130) for detecting the concentration and distribution of smoke using the principle of light scattering by smoke particles suspended in the air. The fire integrated control server (200) includes an AI analysis unit (210) for performing spatiotemporal alignment to align data with different resolutions and sampling periods, including visible light images, infrared thermal images, and scattered light sensor data, into the same time axis and coordinate system, and immediately triggering an alarm upon fire determination and An AI-based multispectral image fusion type fire monitoring system is provided, characterized by including a field control unit (220) for transmitting a signal to a fire integrated control server (200), an integrated management interface (230) for visualizing collected images and analysis results and providing them to a terminal (300), a remote response instruction module (240) for managing reporting to relevant agencies and remote control of on-site fire extinguishing equipment by combining the judgment result of the AI ​​analysis unit (210) and the final approval of the manager through the integrated management interface (230), and a big data learning module (250) for accumulating and analyzing all detection data and judgment history occurring within the fire integrated control server (200). Effects of the invention

[0011] The effects of the present invention are as follows.

[0012] First, through multi-sensor data fusion, it provides robust sensing performance against weather conditions or changes in the surrounding environment.

[0013] Second, through the analysis of the flame's fractal contours and infrared temperature rise patterns, it is possible to clearly distinguish phenomena similar to actual fires.

[0014] Third, the organic connection between the field control unit and the central control server enables integrated fire management across a wide area. Brief explanation of the drawing

[0015] FIG. 1 is a drawing showing the overall configuration of the present invention. FIG. 2 is a configuration diagram showing the detailed configuration of the composite camera unit (100) of the present invention. FIG. 3 is a configuration diagram showing the detailed configuration of the fire integrated control server (200) of the present invention. FIG. 4 is an example of operation according to the system of the present invention. Specific details for implementing the invention

[0016] Hereinafter, various embodiments are described in more detail with reference to the attached drawings. The embodiments described in this specification may be modified in various ways. Specific embodiments may be depicted in the drawings and described in detail in the detailed description. However, specific embodiments disclosed in the attached drawings are intended only to facilitate understanding of various embodiments. Accordingly, the technical concept is not limited by specific embodiments disclosed in the attached drawings, and it should be understood that it includes all equivalents or substitutions that fall within the spirit and scope of the invention.

[0017] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but these components are not limited by the aforementioned terms. The aforementioned terms are used solely for the purpose of distinguishing one component from another.

[0018] Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0019] The predefined operating rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined operating rules or artificial intelligence models configured to perform a desired characteristic (or objective) are created by a basic artificial intelligence model being trained using a number of training data by a learning algorithm. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0020] An artificial intelligence model can be composed of multiple neural network layers. Each of the multiple neural network layers has multiple nodes and weight values, and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, multiple weights can be updated so that the loss value or cost value obtained by the artificial intelligence model during the learning process is reduced or minimized. Additionally, to minimize the loss value or cost value, multiple weights can be updated in a direction that minimizes the gradient associated with the loss value or cost value. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0021] A network is a network that serves as a transmission path for web pages; it may be a closed network such as a LAN (Local Area Network) or WAN (Wide Area Network), but it is desirable for it to be an open network such as the Internet. The Internet refers to a global open computer network structure that provides the TCP / IP protocol and various services existing at its upper layers, namely HTTP (HyperText Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), SNMP (Simple Network Management Protocol), NFS (Network File Service), and NIS (Network Information Service).

[0022] The terminal can be implemented in various forms. For example, the terminal described in this specification may be a mobile terminal such as a smartphone, tablet PC, PDA (Personal Digital Assistant), PMP (Portable Multimedia Player), or MP3 player, as well as a fixed terminal such as a smart TV or desktop computer.

[0023] In this specification, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. When a component is described as being “connected” or “connected” to another component, it should be understood that it may be directly connected to or connected to that other component, or that there may be other components in between. On the other hand, when a component is described as being “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

[0024] Meanwhile, a "module" or "part" for a component as used in this specification performs at least one function or operation. Furthermore, a "module" or "part" may perform a function or operation by hardware, software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts," excluding a "module" or "part" that must be performed on specific hardware or on at least one processor, may be integrated into at least one module. A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0025] In addition, power, power transmission, and control therefor for the following assembly configurations and embodiments, including "by control," follow conventional technology including terminals, applications, hardware control modules, etc., so they are omitted to avoid redundancy.

[0026] In addition, the operation embodiments and configurations described in a general manner without being explained in detail below follow the prior art and are omitted in order to focus on describing the purpose of the present invention and the resulting effects.

[0027] Furthermore, in describing the present invention, if it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the essence of the invention, such detailed description is abbreviated or omitted.

[0029] The present invention relates to a system for predicting the location of a fire and the direction of its spread in real time by simultaneously collecting visible light images, infrared thermal images, and smoke diffusion pattern images to detect flames, smoke, and heat sources at an early stage, and by fusion analyzing the collected multispectral data using an artificial intelligence (AI) algorithm.

[0031] Referring to FIG. 1, the present invention includes a composite camera unit (100), a fire integrated control server (200), and an administrator terminal (300), and each component can be communicated by a network.

[0033] The specific details of each component are as follows.

[0035] The composite camera unit (100) is an integrated module installed in a fire monitoring target area to collect multispectral data related to fire in real time.

[0036] The above composite camera unit (100) is configured to include multiple sensors detecting different wavelength bands within a single housing or to arrange them adjacently, thereby simultaneously acquiring visible light images, thermal images, and smoke particle data at the same viewing angle. This provides basic data that can fundamentally resolve the problem of false detection caused by weather conditions or surrounding environments (light reflection, water vapor, etc.), which is a limitation of single-sensor-based systems.

[0038] Referring to FIG. 2, the composite camera unit (100) is configured to include a visible light image sensor (110), an infrared sensor (120), and a scattered light detection sensor (130).

[0040] The visible light image sensor (110) is a sensor for acquiring visual information of the scene by capturing an image corresponding to the human visible light range.

[0041] The above visible light imaging sensor (110) is used to capture high-resolution images of whether a flame has occurred, its shape, color, and direction of diffusion. The acquired images are subsequently used by the AI ​​analysis unit (210) to analyze the unique fractal contour shape of the flame, thereby serving as a criterion for distinguishing light reflections from lamps or moving shadows from actual fires.

[0042] The above visible light imaging sensor (110) is intended to visually convey the real-time situation at the scene to the manager in the event of a fire, thereby helping to make an intuitive judgment.

[0044] The infrared sensor (120) detects thermal radiation energy emitted from an object and collects temperature information.

[0045] The infrared sensor (120) is intended to detect potential heat sources that are difficult to identify with the naked eye or temperature rises inside a sealed space. By analyzing the infrared temperature rise pattern at a specific point in a time series, it detects early signs of fire where a 'rapid energy change' occurs, rather than a simple high temperature state.

[0046] The infrared sensor (120) above enables accurate identification of the center of the fire (ignition point) even in environments where it is difficult to obtain visible light due to thick smoke.

[0048] The scattered light detection sensor (130) is intended to detect the concentration and distribution of smoke by utilizing the principle that light is scattered by smoke particles suspended in the air.

[0049] The scattering light detection sensor (130) is intended to capture fine smoke in the early stages of a fire that is difficult to detect with visible light sensors or infrared sensors, and collects images of the smoke particle concentration change rate and smoke diffusion pattern to distinguish between general floating matter such as water vapor or dust and actual fire smoke.

[0050] The scattered light detection sensor (130) provides spatial distribution data of smoke density so that the AI ​​analysis unit (210) can perform time series analysis, thereby enabling the prediction of a false signal of fire occurrence in advance.

[0052] The fire integrated control server (200) is intended to process and judge collected data and manage the entire system.

[0053] Referring to FIG. 3, the fire integrated control server (200) is configured to include an AI analysis unit (210), a field control unit (220), an integrated management interface (230), a big data learning module (240), and a remote response instruction module (250).

[0055] The AI ​​analysis unit (210) is intended to determine whether there is a fire by fusion analysis of spatiotemporal data alignment, the fractal contour of the flame, temperature patterns, etc.

[0056] The above AI analysis unit (210) integrates heterogeneous data collected from the composite camera unit (100) to finally determine whether there is a fire and predict future situations.

[0058] Specifically, the AI ​​analysis unit (210) performs spatiotemporal alignment to align data with different resolutions and sampling periods, such as visible light images, infrared thermal images, and scattered light sensor data, into the same time axis and coordinate system. Through this, changes in brightness, temperature rise, and smoke particle concentration observed at a specific point are integrated into a single dataset to increase the precision of the analysis.

[0059] In addition, the AI ​​analysis unit (210) does not simply determine whether the threshold of an individual sensor is exceeded, but rather analyzes the brightness of the image, the temperature of the heat source, and the concentration pattern of smoke particles in conjunction. By identifying the correlation in which the three indicators change in conjunction, the problem of false detection caused by light reflection, water vapor, shadows, etc., which occurred in a single-sensor-based system can be drastically resolved.

[0061] The intelligent time-series analysis engine provided within the AI ​​analysis unit (210), which is a core component of the present invention, is characterized by matching heterogeneous multispectral data collected from the composite camera unit (100) in real time and determining signs of fire by quantifying them into physical and mathematical indicators.

[0062] The intelligent time series analysis engine analyzes data input from each sensor according to the time series (t) to derive the following key indicators.

[0063] First, a flame candidate region is detected in an image obtained from a visible light imaging sensor (110), and the flame fractal dimension (D_f) is calculated by analyzing the irregular self-similarity of the contour of the region. This serves as a decisive basis for distinguishing between the actual flame and the light reflection or shadow of lighting that has a standardized shape.

[0064] Second, the thermal radiation energy detected through the infrared sensor (120) is analyzed to calculate the temperature rise rate (ΔT / Δt), which is the amount of temperature change above a specific threshold, in real time. Through this, the gradual temperature rise caused by electric heating appliances and the rapid spread of the heat source caused by fire are clearly identified.

[0065] Third, the smoke particle concentration change rate (ΔC / Δt) is derived based on the smoke particle density data measured by the scattered light detection sensor (130). This is used to detect continuous and accelerated smoke generation patterns that occur in the early stages of a fire, unlike water vapor or simple airborne dust.

[0066] This engine does not evaluate extracted individual indicators independently, but performs an integrated analysis through a multi-sensor fusion algorithm.

[0067] Brightness, temperature, and concentration data collected from different sensors are aligned along the same coordinate system and time axis to identify the correlation of complex physical quantities occurring at a specific point (x, y, t). By comparing the analyzed data patterns with an artificial intelligence learning model, false positive factors caused by light reflection, water vapor, shadows, etc., which were problematic in existing single-image-based systems, are drastically eliminated.

[0068] Furthermore, this intelligent time-series analysis engine goes beyond analyzing the current state to predict near-future fire risks based on collected time-series data. By analyzing smoke particle concentration and temperature acceleration patterns, it generates warning signals of a fire outbreak before actual flames become visually confirmed, providing advance notification. It is configured to pinpoint the fire origin in real time while simultaneously combining smoke particle movement vectors with heat source diffusion data to precisely predict the direction and speed of fire spread.

[0070] In other words, the intelligent time series analysis engine of the AI ​​analysis unit (210) calculates the hourly change of data obtained from the scattered light sensor to distinguish whether the diffusion pattern is due to fire or simply floating matter, tracks rapid thermal energy changes in a time series to detect abnormal signs in the early stages of ignition, and analyzes whether the outline of the object detected in the visible light image has irregular and self-similar fractal characteristics unique to flames to prevent misdiagnosis caused by lighting or reflected light.

[0071] In addition, the AI ​​analysis unit (210) operates a pseudo-signal pre-prediction model that calculates the probability that current signs will lead to an actual fire based on accumulated time series data, identifies the location of the fire in real time according to the analysis results, and predicts the direction of spread by simulating the movement path of smoke and flames.

[0073] The fire monitoring system according to the present invention is characterized by simultaneously ensuring the accuracy of detection and the speed of response by processing heterogeneous data collected from a composite camera unit (100) in stages through multi-sensor fusion, AI-based pattern analysis, and a real-time prediction model.

[0074] Individual data collected from a visible light imaging sensor (110) that captures the visible light region, an infrared sensor (120) that detects thermal radiation, and a scattered light detection sensor (130) that detects the distribution of smoke particles are spatially aligned in an AI analysis unit (210).

[0075] This integrates data with different wavelength bands and resolutions into a single coordinate system, enabling changes in brightness, temperature, and smoke concentration at a specific point to be recognized as a single object. This fusion of data levels dramatically reduces false positives caused by light reflection, water vapor, and shadows, which occurred in existing single-image-based surveillance systems, through the mutual verification of physical variables.

[0076] In addition, the fused dataset undergoes a precise pattern analysis process through an intelligent time-series analysis engine. The AI ​​analysis unit (210) tracks the rate of change in smoke particle concentration and the infrared temperature rise pattern in a time series, and specifically analyzes the fractal contour shape of the flame candidate region within the visible light image. This does not simply determine whether a threshold is exceeded, but rather compares and analyzes the irregular self-similarity of the actual flame and the smoke diffusion flow pattern using a learned artificial intelligence algorithm, thereby fundamentally blocking false alarms and definitively determining whether a fire has occurred.

[0077] The determined fire information is not limited to a simple notification but calculates future risk factors through a real-time prediction model integrated within the system. This system identifies the fire occurrence point in real time and simultaneously predicts the direction and speed of fire spread based on analyzed smoke diffusion patterns and heat source change trends. The prediction model is configured to generate a preliminary signal of fire occurrence based on collected time-series data, thereby delivering an early warning to the administrator terminal (300) and the control server before it leads to an actual large-scale fire, thereby securing a golden time.

[0079] The field control unit (220) is intended to immediately generate an alarm and transmit a signal to the fire integrated control server (200) upon fire detection.

[0080] The above-mentioned field control unit (220) receives the analysis results derived from the AI ​​analysis unit (210), executes immediate initial measures at the site when a fire is confirmed, and is the subject of communication and execution that organically exchanges data with the upper server, the fire integrated control server (200) and the administrator terminal (300).

[0081] As soon as it is determined to be an actual fire through the AI ​​analysis unit (210), the field control unit (220) directly controls the field devices by minimizing network delay time. In addition, it activates sirens, warning lights, or voice guidance broadcasting devices installed at the site to induce field personnel to evacuate immediately or engage in initial suppression, and depending on the settings, it can cut off the power to the fire zone or automatically perform physical spread prevention measures in conjunction with firefighting equipment (sprinklers, fire shutters, etc.).

[0082] The above-mentioned field control unit (220) transmits emergency situations occurring at the site to a higher-level control system without delay to enable wide-area response, and transmits fire judgment signals generated by the AI ​​analysis unit (210) and predicted fire grade information to the fire integrated control server (200). Through this, not only simple alarm signals but also visible light / infrared image fragments that caused the fire and temperature / smoke concentration values ​​at the time are transmitted in the form of metadata, allowing the manager to accurately understand the situation.

[0083] In addition, the field control unit (220) includes a function to maintain the reliability of the system even in harsh field environments or network failure situations. It is equipped with a safe mode that performs independent field alarms and control based on local data from the composite camera unit (100) and the AI ​​analysis unit (210) even if communication with the upper server is temporarily disconnected, and constantly checks the normal operation and communication status of each sensor within the composite camera unit (100) to immediately report to the integrated management interface (230) in the event of an abnormality.

[0085] The integrated management interface (230) is intended to visualize the collected images and analysis results and provide them to the terminal (300).

[0086] The integrated management interface (230) provides a user interface (GUI) environment that visualizes source data collected from the composite camera unit (100) and high-dimensional analysis results derived from the AI ​​analysis unit (210) and transmits them to the administrator terminal (300).

[0088] The above integrated management interface (230) outputs visible light images, infrared thermal images, and scattered light diffusion pattern images individually, or provides them as a single integrated image by overlapping them. Through this, the manager can visually and immediately identify the ignition point of the fire and the heat source distribution even in a smoke-filled environment.

[0090] In addition, the integrated management interface (230) displays the operating status, network connection status, and data reception sensitivity of each sensor within the composite camera unit (100) in the form of a real-time dashboard.

[0091] The integrated management interface (230) visualizes the temperature rise rate (ΔT / Δt), smoke concentration change rate (ΔC / Δt), and the fractal dimension (D_f) of the flame calculated by the AI ​​analysis unit (210) as real-time graphs, allowing the manager to intuitively grasp the 'trend' of fire precursor phenomena rather than simple numerical values. By overlaying the expected spread path and danger zone centered on the analyzed fire location onto a map (GIS) or plan view as a translucent layer, it helps determine the priority of initial measures.

[0093] Additionally, the integrated management interface (230) streams analysis results and video in real time with a resolution optimized for the type of administrator terminal (300) (PC, tablet, smartphone, etc.) and network bandwidth. Normally, it monitors the entire area in rotation, but when a risk sign exceeding a threshold set by the AI ​​analysis unit (210) is detected, it immediately highlights the video of the area and generates an alert pop-up to alert the administrator.

[0095] The above integrated management interface (230) records the fire sign data, the basis for AI judgment, and the manager's response history in a timeline format by linking them with a database. This can be used as basic data for the big data learning module (240) in the future and can also be automatically converted into an accident analysis report.

[0097] The remote response instruction module (240) is intended to manage the reporting to relevant agencies and the remote control of on-site firefighting equipment by combining the judgment result of the AI ​​analysis unit (210) and the final approval of the manager through the integrated management interface (230).

[0098] When a fire confirmation signal is generated by the AI ​​analysis unit (210), the above remote response instruction module (240) immediately transmits an approval request notification to the administrator terminal (300). The moment the administrator checks the situation and operates the response button, it enters the real-time response phase according to a predefined protocol.

[0099] In addition, the remote response instruction module (240) can be configured to switch to an automatic mode that executes an immediate response process without administrator approval when the probability of fire determined by AI is extremely high, in preparation for urgent situations or when an administrator is absent.

[0101] The above remote response instruction module (240) automatically transmits the accurate GPS location information of the fire occurrence point, the fire grade, and a video clip of the scene situation to the fire station or a related security company as soon as the administrator's approval is completed. By establishing a reporting system based on digital data rather than a simple telephone reporting method, the efficiency of the initial response is maximized by disseminating the situation in real time to multiple stakeholders (firefighters, site managers, etc.) via mobile push or SMS.

[0103] The above remote response instruction module (240) directly operates fire extinguishing equipment (sprinklers, gas fire extinguishing equipment, fire shutters, etc.) at the site from a remote location. This enables physical suppression in the early stages of ignition when it is dangerous for on-site personnel to approach. It secures a safe evacuation route for occupants by controlling the automatic opening of doors inside the building or the illumination of emergency guidance lights. In particular, it can instruct the transmission of information on the optimal evacuation route that avoids the direction of fire spread predicted by AI, in conjunction with on-site guidance broadcasting equipment.

[0105] The above remote response instruction module (240) records all initial response processes on a timeline, such as the time of approval by the manager, the time of sending the report, and whether the remote control command was issued and executed. This record is used not only to generate a post-accident analysis report but is also transmitted to the big data learning module (250) and used as statistical data to improve response efficiency.

[0107] The big data learning module (250) is a self-evolving component that accumulates and analyzes all detection data and judgment history generated within the fire integrated control server (200), continuously advances the judgment algorithm of the AI ​​analysis unit (210), and improves the reliability of the system.

[0109] The above big data learning module (250) stores raw data, such as visible light images, infrared heat maps, and scattered light concentration values ​​collected from the composite camera unit (100), according to timelines before and after the occurrence of a fire. In addition, the AI ​​analysis unit (210) matches the data determined to be a fire with the results confirmed at the actual site (whether it is an actual fire) to generate ground truth and classify it into a learning dataset.

[0111] The above-mentioned big data learning module (250) intensively analyzes false alarm cases to reduce false alarms, which were a chronic problem of the existing system. To this end, it separately extracts cases that have patterns similar to fire, such as light reflection, water vapor, and shadows, but are determined not to be fire, and re-analyzes the rate of change in smoke particle concentration or the fractal contour characteristics of the flames appearing in the false alarm cases to refine the feature value extraction logic so that the AI ​​analysis unit (210) can distinguish them more clearly from actual fires.

[0113] The above big data learning module (250) periodically retrains deep learning models such as CNN (convolutional neural network) or RNN / LSTM based on accumulated big data, derives weights optimized for the seasonal characteristics of the site or the installation environment (lighting, structures, etc.), and updates the AI ​​analysis unit (210) so that over time, it provides customized detection performance optimized for the installation environment. Before applying the updated model to the site, a virtual test is performed based on the previously accumulated data to verify the reduction in false positive rates and the improvement in detection speed, and the trend of system performance improvement is generated as statistical data and reported to the manager terminal (300) held by the manager through the integrated management interface (230).

[0115] The administrator terminal (300) is a physical device (PC, smartphone, tablet, etc.) that allows an administrator to monitor real-time situations and take remote actions, and can display an integrated management interface (230) on the screen and generate an input signal for remote response instructions and transmit it to the fire integrated management server (200).

[0117] Referring to Fig. 4, the operation process of the present invention is explained as follows.

[0119] S1) Field Information Input Step

[0120] The above-mentioned field information input step (S1) is a step of collecting the physical condition of the fire monitoring target area from various angles through the composite camera unit (100).

[0121] That is, the visible light imaging sensor (110) acquires a visual image of the site to determine the presence and shape of the flame, and the infrared sensor (120) detects thermal radiation energy emitted from the object to collect data on the invisible potential heat source and temperature distribution, and the infrared sensor (120) detects thermal radiation energy emitted from the object to collect data on the invisible potential heat source and temperature distribution.

[0122] In step S1, data from three heterogeneous sensors are collected simultaneously to form the basic dataset for analysis.

[0124] S2) Fire risk analysis stage

[0125] The fire possibility analysis step (S2) is a step for intelligently processing the data collected through the above-mentioned site information input step (S1) by the AI ​​analysis unit (210).

[0126] The AI ​​analysis unit (210) maps the complex state of a specific point by matching visible light, infrared, and smoke concentration data in space and time. It does not simply check for exceedance of threshold values ​​of individual values, but rather analyzes changes in image brightness, rapid temperature rise patterns, and the rate of change in smoke particle concentration in conjunction. In addition, it analyzes the unique fractal contour shape of the flame to drastically eliminate false detection elements caused by light reflection, water vapor, shadows, etc., confirms whether there is an actual fire, and operates a model that predicts the location of the fire and the future direction of spread based on the current state.

[0128] S3) On-site Action Step

[0129] The above-mentioned field action step (S3) is a step for the field control unit (220) to execute an immediate initial response when the risk level resulting from the fire analysis exceeds the judgment criteria.

[0130] In other words, it is intended to control on-site sirens, flashing lights, or emergency broadcasting equipment to notify personnel inside of dangerous situations, and to secure the golden time to minimize casualties by sounding an alarm through edge-based judgment at the site before waiting for commands from the upper server.

[0132] S4) Analysis result transmission step

[0133] The analysis result transmission step (S4) is a step for transmitting the analysis results to the administrator terminal (300) through the integrated management interface (230). That is, step S4 is a step in which all analysis data is reported to the upper layer for precise monitoring simultaneously with on-site action.

[0134] In step S4, the integrated management interface (230) generates a fused image in which visible light images and thermal images are combined, as well as AI analysis indicators (such as a temperature change graph) in real time, and the generated dashboard-type information is immediately transmitted to a manager terminal (300) via a network, allowing the manager to precisely monitor the field situation in real time even from a remote location.

[0136] S5) Remote response step

[0137] The remote response step (S5) is a step in which, based on the analysis results transmitted in the analysis result transmission step (S4), when an administrator issues a command through the administrator terminal (300), the remote response instruction module (240) performs inter-agency linkage, etc.

[0138] In step S5, when the administrator operates the approval button through the terminal (300), the remote response instruction module (240) is activated and automatically transmits the exact location of the fire and the scene video to the fire station or security control center to request emergency dispatch. Additionally, if necessary, active response measures such as remotely activating fire extinguishing equipment or opening doors can be instructed.

[0140] S6) Data learning step

[0141] The data learning step (S6) is a step in which data from all processes is stored in the big data learning module (250) to enhance system performance.

[0142] That is, the system improves its performance on its own after the situation ends or during continuous operation. In the data learning stage (S6), all fire sign data and response history that occurred are stored in the big data learning module (250). In particular, by analyzing 'false positive cases' and retraining the AI ​​model, the accuracy of judgment when similar situations occur is increased, and the reliability of the entire system is improved by strengthening customized pattern analysis capabilities that reflect the unique environmental characteristics of the site (special lighting, ventilation system, etc.).

[0144] The fire integrated control server (200) further includes a dynamic insurance rate calculation module (260), wherein the dynamic insurance rate calculation module (260) combines fire monitoring data and financial indicators to quantify the risk at the site based on the real-time fire occurrence probability transmitted from the AI ​​analysis unit (210) and the expected damage severity calculated through the data of the scattered light detection sensor (130) to derive a preliminary risk rate, and calculates a compensation coefficient for the preliminary risk rate based on the accumulated accident-free operation time data stored in the big data learning module (250) to generate a final dynamic rate.

[0145] In addition, the dynamic insurance rate calculation module (260) normalizes the real-time fire occurrence probability and the cumulative accident-free operation time to a value between 0 and 1 according to a system standard scale to synchronize the operation units between the data, and calculates a safety compensation coefficient by applying a logarithmic decay model to the cumulative accident-free operation time to apply a weighted discount range to the long-term accident-free maintenance period compared to the accident-free period at the beginning of operation, calculates a system reliability coefficient by combining the sensor alignment accuracy of the composite camera unit (100) and the false positive removal rate of the big data learning module (250), and performs a correction operation by subtracting the safety compensation coefficient from the preliminary risk rate and dividing the derived value by the system reliability coefficient, thereby controlling the final dynamic rate so that the higher the monitoring precision of the system, the lower the final dynamic rate.

[0147] The dynamic insurance rate calculation module (260) is a fintech combined configuration that calculates fire insurance rates for a monitored area in real time or periodically by combining real-time risk indicators calculated by the AI ​​analysis unit (210) and safety history accumulated in the big data learning module (250).

[0148] The dynamic insurance rate calculation module (260) receives the real-time fire occurrence probability (P) and estimated damage scale (S) from the AI ​​analysis unit (210) and quantifies the current physical risk status. Additionally, through multi-sensor fusion analysis, as the false positive rate decreases, the system reliability weight is increased and utilized as an insurance premium discount factor (safety compensation coefficient). Furthermore, it collects data on accident-free periods and preventive measure implementation rates (D) from the big data learning module (250) to evaluate the level of long-term risk management.

[0150] The dynamic insurance rate calculation module (260) according to the present invention is linked with the AI ​​analysis unit (210) and the big data learning module (250) and performs the function of converting the physical risk status of a fire monitoring target into a financial value.

[0151] This module operates according to the following temporal and logical sequence from data collection to the final dynamic rate calculation.

[0153] 1) Data Collection and Normalization Step

[0154] First, real-time fire occurrence probability data is received from the AI ​​analysis unit (210), and cumulative accident-free operation time data is collected from the big data learning module (250) since the last accident or risk sign was detected. The collected data is first normalized to a value between 0 and 1 according to the system's standard scale.

[0156] This step involves the process of converting heterogeneous data with different physical units into common indicators suitable for calculating financial rates. Based on the statistical range of the collected source data, the system performs normalization through the following procedure.

[0157] The fire occurrence probability data (factor A) is converted into a numeric value in the form of a decimal point (0.0 to 1.0) between 0% and 100% derived from the AI ​​analysis unit (210). At this time, 0% is the minimum risk (0), and the maximum alarm threshold set by the system (e.g., confidence level of 95% or higher) is mapped to the maximum risk (1) to adjust the sensitivity of the analysis.

[0158] The accumulated accident-free operating time data (factor C) quantifies the 'hour' unit data collected from the big data learning module (250). At this time, the maximum value used as a standard is set by the manager based on the characteristics of the facility or the insurance renewal cycle (e.g., 1 year or 8,760 hours).

[0159] First, check if the collected source data is within a predefined valid range. If the accident-free operation time exceeds a preset maximum threshold (e.g., 1 year), forcibly fix it to the maximum value of 1.0 to prevent data distortion.

[0160] For data within the valid range, the current value is placed on a virtual coordinate axis with a lower limit of 0 and an upper limit of 1. For example, if the standard upper limit for accident-free operation time is 100 days and the current accumulated time is 50 days, this is converted to a neutral value of 0.5.

[0161] Normalized probability data and time data are aligned in the same floating-point format to synchronize them, preventing computational errors between the data during the subsequent step of 'applying safety compensation factors'.

[0162] An example of this is when the AI ​​analysis unit (210) determines the current situation as a fire with an 80% probability (A=0.8) and only 10 hours (B=approx. 0.001) have passed since the last accident. The two normalized indicators have a high risk level and a very low safety compensation factor, resulting in a steep increase in the final dynamic rate.

[0163] Another example is when the probability of fire is extremely low at less than 5% (A = approximately 0.05) and the accident-free operating time reaches 6 months (B = 0.5), which is half of the standard upper limit of 1 year. In this case, the normalized safety indicator (B) overwhelms the risk indicator (A), and thus receives a high weight at the insurance rate discount stage.

[0166] 2) Preliminary risk rate calculation step

[0167] The preliminary risk rate is calculated by multiplying the system-predicted fire spread range (estimated damage area) by the normalized probability of fire occurrence. In this process, the cumulative sum of values ​​over a set monitoring period is used instead of one-time data from a specific point in time to prevent rate distortion caused by temporary numerical fluctuations.

[0169] This step involves combining normalized individual data over time to quantify the actual risk exposure (preliminary risk rate). To prevent abrupt fluctuations in rates caused by one-off data and to ensure system stability, the process proceeds through the following specific procedures.

[0170] The calculation of preliminary risk rates in this stage is performed in the following order, taking into account temporal continuity.

[0171] First, the system sets a 'monitoring cycle' (e.g., 300 seconds or a specific number of frames), which is the minimum unit of time for evaluating risk. This acts as a filtering mechanism to prevent sudden spikes in values ​​caused by light reflection or water vapor from being immediately reflected in insurance rates.

[0172] Second, the 'normalized fire occurrence probability' values ​​input from the AI ​​analysis unit (210) during the set monitoring period are accumulated and summed in chronological order. That is, at each sampling point, the normalized fire occurrence probability and the expected damage range are first multiplied to obtain the 'instant risk,' and this is accumulated and summed over the set monitoring period to be averaged.

[0173] Third, the 'fire spread range (estimated damage area)' predicted in real-time by the system is multiplied by the sum of the accumulated probabilities. This is intended to assign a higher risk score to areas predicted to have a faster spread speed or a larger damage area, even if the probability of fire occurrence is the same.

[0174] Fourth, the value calculated in the above process is averaged by dividing it by the total time of the corresponding monitoring cycle, or compared with a pre-set risk upper limit value to determine the final preliminary risk rate figure.

[0176] The cumulative fire probability sum enhances data reliability by utilizing area data over a specific period rather than point data at a particular time. This setting is designed to offset short-term noise and distinguish only continuous signs of fire.

[0177] The fire spread range weighting is based on the smoke movement vector derived from the scattered light detection sensor (130). Since a wide spread range implies a high likelihood of a major accident if initial suppression fails, this is included as a key weighting factor in the rate calculation.

[0179] This module does not stop at simply summing values, but selects the final result by evaluating the following multiple conditions.

[0180] Within the monitoring period, it is classified as a ‘high-risk exposure’ only if (1) the average fire probability is above the threshold (e.g., 0.6) and (2) the expected damage area is within the top 10% of the total managed area.

[0181] Values ​​that rise sharply for only 1 to 2 seconds within the monitoring cycle and then disappear are considered to be temporary sensor errors or noise caused by external light sources and are excluded from the cumulative summation. This maintains the objectivity of rate calculation.

[0182] In this case, an increase in the cumulative sum indicates that signs of fire are not temporary but are occurring 'continuously'. This serves as grounds for the system to confirm that the situation is an actual fire, and it leads to an upward adjustment of insurance rates to prepare for the risk.

[0183] The spread range weighting is reduced when there are signs of fire but the spread is very slow or limited to a localized area. In such cases, the increase in the preliminary risk rate is limited to prevent unnecessary premium hikes.

[0184] For example, when the monitoring cycle is set to 5 minutes (300 seconds), a situation occurs where the fire probability is 0.01 (very low risk) for 290 seconds, but then spikes to 0.9 (high risk) for 10 seconds due to a sudden light reflection. If this were one-time data, the rate would have skyrocketed, but since this system uses the average after cumulative summing over 300 seconds, the final preliminary risk rate maintains an extremely low value of 0.04.

[0185] Unlike conventional thresholding methods that use only a single condition, this cumulative summation and range weighting method simultaneously considers the persistence and impact of risk. As a result, it significantly reduces fluctuations in false positive rates caused by environmental noise and enables precise risk assessment in actual risk situations. Consequently, it provides reliable financial indicators to insurers and managers, thereby maximizing the economic and technical utility of the entire system.

[0187] 3) Step of applying safety compensation factor

[0188] A safety compensation factor based on accident-free operating time is subtracted from the preliminary risk rate. A logarithmic decay model is applied to accident-free operating time to determine the rate discount, giving higher weight to long-term accident-free periods than to early accident-free periods.

[0190] Specifically, the dynamic insurance rate calculation module (260) does not simply arithmetically sum the 'accident-free operating time,' which is the period during which no fire occurred, but calculates the safety compensation coefficient by applying a logarithmic function attenuation model.

[0191] The technical reason for adopting a logarithmic functional model is that existing linear calculation methods (e.g., assigning 1 point per day) have limitations in evaluating safety values ​​equally during the initial operation period (short-term) and the stabilization period (long-term). However, this invention is based on the observation that the reliability of a fire monitoring system is verified 'probabilistically' as time passes.

[0192] Accident-free operation during the initial stages of operation may be due to simple coincidence or temporary environmental factors. The fact that accident-free operation has been maintained for a long period technically proves that the sensors of the composite camera unit (100) are operating stably without false positives even amidst various environmental changes (season, change in illumination, dust, etc.) and that the manager's safety management system is organically combined with the hardware. Therefore, the longer the accident-free period, the more weight is given to the 'qualitative value of time' to determine the rate discount range.

[0193] The system sets a range in which the reliability index increases rapidly as the accident-free period (t) increases. During the initial stage of operation (e.g., within one month), a conservative safety factor is applied to minimize the fluctuation of the discount rate. From the point when the accident-free period exceeds the 'stabilization threshold' (e.g., cumulative 1,000 hours) set by the system, the weight of the safety reward points awarded per additional accident-free hour is set high. This is because it means that the system has completely overcome and optimized the special noise (water vapor, light reflection, etc.) of the site through the big data learning module (250).

[0195] The final safety compensation factor is calculated through a technical procedure following the chronological order.

[0196] First, the point in time when the final fire judgment signal or danger sign detection signal is generated from the big data learning module (250) is set as the reference point, and the accumulated accident-free operation time up to the current point in time is quantified in minutes. The collected accumulated time is converted by inputting it into a logarithmic function model. At this time, a normalization process is performed so that the value immediately after operation (when time is 0) does not become negative by adding a constant 1 to the time data and taking the logarithm value. That is, since log 0 is undefined, this is to secure the minimum value.

[0197] The final safety compensation coefficient is derived by multiplying the converted value by the 'field management index' and 'equipment normal operation rate' set by the system. If there is a history of sensor errors being reported by the self-diagnosis function of the field control unit (220) during the accident-free period, the weight of that period is lowered by a pre-set penalty ratio (e.g., 0.7 times). Only when the calculated safety compensation coefficient exceeds a reference value (e.g., 0.5) is the final insurance rate determined by deducting that value from the preliminary risk rate.

[0199] An example of this is as follows.

[0200] -Situation A (Initial): If the accident-free period is 10 days, the discount based on the log transformation value is very minimal and remains close to the base rate. This means that system stability has not yet been verified.

[0201] -Situation B (Long Term): If the accident-free period increases to 100 days under the same conditions, a much larger discount factor is calculated by combining the log curve and weighting factors, rather than simply being 10 times that of Situation A.

[0203] This configuration encourages the manager to prevent system malfunctions and maintain long-term stability, and consequently creates a virtuous cycle structure that proves the precision of the AI ​​analysis unit (210) with actual field data.

[0205] 4) System reliability-based calibration step

[0206] The calculated preliminary rate is divided by the system reliability index, which is defined by the sensor alignment accuracy and false positive removal rate of the composite camera unit (100), to perform final correction. This is designed so that the risk index is evaluated lower as the system operates more precisely (the higher the reliability).

[0208] The preliminary risk rate is an intermediate output derived from the cumulative sum of the fire occurrence probability, spread range, and monitoring period in the previous stage. This figure reflects purely the 'risk at the site'.

[0209] The system reliability index is a value that combines the physical condition of the composite camera unit (100) and the judgment accuracy of the AI ​​analysis unit (210). This is an indicator of how reliable the risk data derived by the system is.

[0210] The final dynamic rate is calculated through the following procedure.

[0211] First, the system first collects a 'sensor matching accuracy' value, which indicates the degree of pixel matching between sensors, from the composite camera unit (100), and collects a 'false positive removal rate' value, which is the matching rate of the actual situation compared to past judgments, from the big data learning module (250).

[0212] Second, the two collected factors are combined to generate a 'System Reliability Index'. At this time, the reliability index is designed to have a value greater than 1 based on a default value of 1, as matching accuracy and false positive removal rates improve. (e.g., 1.5 for top-tier performance, 1.0 for standard performance, 0.7 for degraded performance, etc.)

[0213] Third, perform the operation of dividing the previously derived preliminary risk rate by the above-mentioned system reliability index.

[0214] Fourth, the final dynamic rate derived from the division result undergoes final filtering so that it does not exceed the minimum / maximum rate range set by the manager, and then outputs it to the integrated management interface (230).

[0216] Mathematically, as the denominator of the division (system reliability index) increases, the result (final dynamic rate) decreases. The reason for adopting this logic is as follows.

[0217] As a compensation system based on precision, even for the same fire risk situation, if the system analyzes the risk with great precision (high reliability), the possibility of false positives is low, so it is reasonable to lower the final risk assessment amount.

[0218] Conversely, if system reliability decreases due to sensor mismatches or a low false positive rejection rate (when the denominator becomes less than 1), the result of the division becomes larger than the reserve rate. This has the effect of additionally reflecting the risk cost of system uncertainty in the insurance rate.

[0219] If the system reliability index is greater than 1 (excellent), the denominator increases, so the final dynamic rate decreases. This implies that "it is being managed more safely than the actual risk thanks to a precise monitoring system," providing the manager with a technical incentive in the form of a rate discount.

[0221] When the system reliability index is less than 1 (insufficient), the denominator becomes smaller, so the final dynamic rate increases. This means that "since there is a high degree of uncertainty in the currently reported risk data due to system obsolescence or the possibility of false positives, one must prepare for the risk through higher premiums."

[0223] For example, if the preliminary risk rate is 80 units and the system reliability index is 1.25 (bonus weighting given for excellent sensor matching and false positive removal performance), the final dynamic rate becomes '80 / 1.25 = 64 units (approximately 20% discount effect)'.

[0224] Unlike conventional technology that provided only a single risk value, this division-based correction logic simultaneously reflects the magnitude of the risk and the accuracy of the measurement in the financial rate. This provides a strong technical incentive for the manager to always maintain the sensor status of the composite camera unit (100) at its best and continuously update the AI ​​analysis model, resulting in the effect of raising the reliability and safety of the entire fire monitoring system.

[0226] 5) Final dynamic rate determination and output stage

[0227] The final dynamic rate derived through the above steps is applied to the standard insurance rate and transmitted to the administrator terminal (300) and the external financial server.

[0229] The key factors used in the rate calculation logic of this system and the basis for their settings are as follows.

[0230] - Real-time Fire Occurrence Probability (Factor A): This quantifies the extent to which the fractal contour analysis results of the visible light image and the infrared temperature rise pattern simultaneously indicate signs of fire. This is a factor that directly reflects the 'likelihood' of a fire occurring.

[0231] - Estimated damage depth (Factor B): This refers to the predicted fire spread range based on the movement vector of smoke particles detected by the scattered light detection sensor (130) and airflow data. Even if the probability is the same, if the estimated scale of damage is large, the risk is high, so it is set as a weighting factor.

[0232] - Accident-free operation time (Factor C): This is the cumulative time during which normal operation was maintained after the detection of the last signs of danger. The reason for processing this as a logarithmic function is that a long-term history of safe operation provides more objective proof of fire prevention management capabilities than a short period of safety immediately following an accident.

[0233] - System Reliability Coefficient (Factor D): This quantifies the frequency of agreement among the analysis results of the three sensors—visible light, infrared, and scattered light—and the validity of the false positive elimination algorithm. Designed so that the denominator increases as the accuracy of multi-sensor fusion increases, it serves as the basis for lowering insurance premiums in areas where precise surveillance is conducted.

[0235] Numerical changes in each factor have the following technical and financial implications.

[0236] - Increase in Factor A (probability) and Factor B (depth): As the odds of the two factors increase, the risk weight rises steeply, which leads to an immediate increase in insurance rates. In particular, if Factor B exceeds a pre-set threshold value (e.g., 15% of the managed area), it is classified as a high-risk group and the weight is increased by 1.5 times.

[0237] - Increase in Factor C (Accident-free time): As the accident-free time increases, the discount amount deducted from the total risk increases. For example, when 720 hours (30 days) or more of consecutive accident-free time is achieved, an additional 20% discount weight is applied compared to the existing amount to motivate managers to manage safety.

[0238] - Increase in Factor D (Reliability): If the reliability index of the system exceeds a pre-set threshold value (e.g., weighted average 0.85), the denominator of the total rate calculation is increased to lower the final premium. This is based on the logic that the more accurate a system is and the fewer false positives it has, the lower the actual risk should be assessed.

[0240] An example from an actual operating environment is as follows.

[0241] Assumption: Assume a factory with a standard rate of 100 units, where AI analysis calculates the fire probability to be 0.1 (10%) and the expected damage range to be 0.2 (20%), there have been no accidents for 30 days, and the system reliability is 0.9.

[0243] Output process:

[0244] The risk index is proportional to the product of the probability (0.1) and the damage range (0.2).

[0245] Here, the safety compensation factor (e.g., 0.05) based on the accident-free period (30 days) is subtracted.

[0246] The result is corrected by dividing it by the system reliability (0.9).

[0248] Interpretation: In this case, because the accident-free period is long and the system reliability is high, the final dynamic rate can be maintained at a level lower than the standard rate of 100 even if a potential risk (probability of 10%) exists. If the accident-free period is 0 and the reliability is 0.5, the rate will skyrocket to several times the standard rate even with the same probability of fire.

[0250] Unlike conventional technology that simply applies fixed insurance rates, the present invention dynamically calculates rates by mathematically combining real-time risk analysis results based on multi-sensor fusion data (110-130) with time-series safety operation history (250). This configuration provides a technical incentive for managers to increase system reliability and maintain an accident-free period, and consequently yields a chain of technical effects that improve the prevention of false positives and the precision of fire prevention.

[0252] The present invention further comprises an environmental sound collection unit (400) and an irregular acoustic gap analyzer (410), wherein the environmental sound collection unit (400) is intended to collect physical sound data of a fire monitoring target area in a broadband and supply it to a system. The environmental sound collection unit (400) is configured to complement fire precursor phenomena that are difficult to capture by visible light images or infrared sensors from an acoustic perspective.

[0253] The above-mentioned environmental acoustic collection unit (400) includes a high-sensitivity microphone capable of detecting frequencies ranging from audible frequencies (20Hz to 20kHz) to inaudible ultrasonic ranges so as to accommodate all minute vibration sounds, ecological noises, and mechanical operating sounds at the site. After converting the collected analog acoustic data into a digital signal, it performs a preprocessing process to remove temporary impact sounds or white noise unrelated to fire. The data thus refined is converted into energy density values ​​by frequency band, rather than a simple recording file, and is transmitted to an unstructured acoustic gap analyzer (410), which is a sub-component of the environmental acoustic collection unit (400).

[0256] The environmental sound collection unit (400) is for collecting and preprocessing sound data of the monitored area in a broadband manner, and

[0257] The atypical acoustic gap analyzer (410) is intended to analyze acoustic data received from the environmental acoustic collection unit (400) to detect a rapid decrease in acoustic energy due to fire precursors.

[0258] The above-described atypical acoustic gap analyzer (410) is characterized by being configured to detect an acoustic gap section in which sound energy rapidly decreases in a specific frequency band by comparing a background energy reference value (E_base), which is the average energy density per frequency band during a preset period, with a real-time energy measurement value (E_real) measured in real time, and to calculate an acoustic gap index based on the scale of the detected acoustic gap and transmit it to the AI ​​analysis unit (210) to adjust the analysis weight for determining the occurrence of a fire.

[0260] The unstructured acoustic gap analyzer (410) is an intelligent engine that analyzes acoustic data received from the environmental acoustic collection unit (400) and captures acoustic gaps, which are peculiar phenomena that appear in the ecosystem or facility just before a fire occurs.

[0261] This module analyzes the meaning of acoustic gaps according to the following temporal and logical order.

[0263] 1) Background noise profiling step

[0264] First, acoustic patterns that occur routinely in the pre-configured area (e.g., insect chirping, regular operating sounds of machinery, etc.) are learned by frequency band to establish a reference average energy density.

[0265] This step is the process of establishing criteria for a 'normal state' to determine fire precursors.

[0266] The collected broadband acoustic data is divided into multiple frequency bands using the Fast Fourier Transform (FFT). At this stage, a log-scale or Mel-scale filter bank can be applied to precisely analyze the bands where specific insect calls or equipment operation sounds are concentrated.

[0267] Data is collected for at least 24 hours, taking into account periodic noise occurring in the area (such as differences in ecological noise between day and night). A moving average algorithm is applied to the collected energy density for each band to set the background energy reference value (E_base). Here, to reflect changes in background noise due to seasonal changes or weather conditions (rain, strong winds, etc.), the background energy reference value (E_base) is automatically updated periodically during normal conditions when there is no fire signal to maintain the validity of the reference.

[0269] 2) Real-time spectrum comparison step

[0270] The power spectral density (PSD) of the sound currently being collected is compared in real time with the above reference value.

[0271] This step involves identifying variability by comparing real-time acoustic signals from the field with a reference profile.

[0272] In the second step, the digital signal preprocessed in the environmental acoustic collection unit (400) is analyzed in real time to calculate the real-time energy measurement value (E_real) at the current time in the form of power spectrum density.

[0273] In addition, the difference between the band-specific background energy reference value (E_base) set in the first step above and the current real-time energy measurement value (E_real) is calculated. This calculates the ratio of the current value to the reference value or the logarithmic attenuation ratio (dB), rather than the difference in simple absolute values, to sensitively detect minute energy losses.

[0275] 3) Energy loss point detection step

[0276] It detects sections where sound energy decreases rapidly in specific frequency bands. This is a process of finding 'atypical silence' that occurs when ecosystems stop activity after detecting minute vibrations from a fire or changes in air density, or when the operating efficiency of mechanical equipment changes due to signs of overheating.

[0278] This step is the process of identifying 'atypical silence' sections where specific frequency bands disappear from normal background noise.

[0279] In step 3, sections where energy in a specific frequency band being compared drops sharply below a preset attenuation threshold (e.g., -20dB or less relative to the reference) are detected, and it is identified whether the detected sections are ecological elements (e.g., insect sounds in the high-frequency band) or mechanical elements (e.g., motor driving sounds in the low-frequency band). That is, the nature of the lost sections is classified by referring to preset ecological frequency databases (e.g., frequency bands of cicadas and birds) and mechanical frequency databases (e.g., frequency bands of fans and motors).

[0280] Since precursor vibrations of fire or minute atmospheric pressure changes cause the cessation of ecosystem activity, points where energy in these specific bands dissipates are designated as candidate sites for acoustic voids.

[0282] 4) Acoustic Gap Index Calculation Step

[0283] The 'acoustic gap index' is derived by quantifying the width and depth of the detected loss segments.

[0285] This step involves quantifying the detected silence phenomenon and converting it into objective risk indicators.

[0286] Specifically, the magnitude of the gap is calculated by multiplying the range of the lost frequency band by the magnitude of the energy reduction, and an acoustic gap index is derived by multiplying the bandwidth weight of the lost frequency band by the energy deviation ratio obtained by dividing the deviation between the background energy reference value (E_base) and the real-time energy measurement value (E_real) by the background energy reference value (E_base). This value is normalized to a value between 0 (normal) and 1 (absolute silence), and the closer the index is to 1, the more completely the noise in the corresponding band has been lost.

[0287] Even if the calculated index exceeds the threshold value (e.g., 0.7), it is finally checked whether the state is maintained for a duration threshold (e.g., 3 seconds) to distinguish it from temporary noise blocking.

[0289] 5) Fusion Analysis Feedback Stage

[0290] If the derived acoustic gap index exceeds a preset threshold, it is transmitted to the AI ​​analysis unit (210) to dynamically adjust the analysis weight for determining the occurrence of a fire.

[0292] Specifically, when a valid acoustic gap is identified, the analysis weight of the environmental acoustic data within the AI ​​analysis unit (210) is immediately increased, thereby controlling the system to determine early as a 'fire precursor situation' even before the data from the visible light image or infrared sensor reaches a threshold.

[0294] Here, the analysis weight refers to the relative importance of the visible light, infrared, and scattered light data of the composite camera unit (100) and the acoustic data of the environmental acoustic collection unit (400) in contributing to the final fire determination. Normally, the system is set so that the sum of all analysis weights is 1, and controls the data of each sensor to be processed evenly or according to a pre-set standard weight.

[0295] If the acoustic gap index derived from the unstructured acoustic gap analyzer (410) exceeds a threshold, the AI ​​analysis unit (210) resets the analysis weight in the following steps.

[0297] If an acoustic gap occurs, it is highly likely to be a fire precursor situation that has not yet been visually confirmed, so the analysis weight of the acoustic data is immediately increased in the overall judgment logic.

[0298] By increasing the analysis weight of infrared thermal map data corresponding to the point where an acoustic precursor signal is generated, control is implemented so that even minute temperature changes are immediately reflected in the fire probability.

[0299] On the other hand, if the acoustic gap index is low but signs of fire are visible only in the visible light image (e.g., lighting reflections), the analysis weight of the image data is temporarily lowered, or the false positive status is determined through cross-validation with the acoustic data.

[0302] This module defines and calculates the state in which the energy of a specific frequency band drops sharply beyond statistical significance as the 'acoustic gap index'.

[0304] The factors for calculating the acoustic gap index are as follows.

[0305] - Background Energy Base (E_base): A value representing the unique acoustic characteristics of the location. It is updated during the initial system installation and through periodic learning, and serves as a comparison point defining the 'steady state'.

[0306] - Real-time energy measurement (E_real): This is the total sum of energy by frequency band observed at the current time.

[0307] - Duration threshold: A minimum duration (e.g., 3 seconds or more) set to confirm that the acoustic gap phenomenon is caused by a fire precursor rather than a simple temporary noise blockage (e.g., a change in wind direction).

[0308] - Acoustic Gap Index (Result): A quantitative risk indicator calculated based on the deviation between the reference value and real-time measurements, and the lost bandwidth.

[0310] If the acoustic gap index exceeds a preset threshold value (e.g., 0.7) and this condition persists for a certain period of time (e.g., 3 seconds) or longer, it is considered a valid 'gap phenomenon'.

[0312] Additionally, if the measured value (E_real) decreases sharply compared to the reference value (E_base), the acoustic gap index rises, which is determined to be a state of 'abnormal silence' where the ecological and mechanical background noise of the site has disappeared.

[0313] If the calculated index result exceeds the threshold value (e.g., 0.7), it is considered a significant sign of danger rather than a simple change in noise. As this value increases, it implies a high likelihood that the ecosystem at the site has detected danger and fled, or that a physical defect in the equipment has occurred.

[0315] An index close to 0 indicates that the noise environment at the site is no different from usual, while an index closer to 1 signifies a state of 'absolute silence' where noise in a specific frequency band has completely disappeared. This technically suggests that the ecosystem has detected danger and ceased activity, or that the efficiency of sound transmission has changed rapidly due to changes in air density caused by fire precursors.

[0317] Assuming the present invention is installed in a mountainous area, it is assumed that the background energy reference value (E_base) of the forest collected through the background noise profiling step (Step 1) was maintained at 0.9. Before a fire occurs, as the ecosystem (insects, birds, etc.) detecting fine smoke and vibrations stops activity all at once, a phenomenon is detected in which the real-time energy measurement value (E_real) drops sharply to 0.1 in a specific frequency band.

[0319] At this time, the acoustic gap index calculated by multiplying the energy deviation ratio (0.8 / 0.9) by the bandwidth weight of the corresponding loss section is calculated to be approximately 0.88. This exceeds the threshold (0.7) pre-set in the system and simultaneously satisfies the duration threshold (3 seconds), thereby generating an immediate acoustic-based fire warning signal.

[0321] As a result, even before the visible light or infrared sensor of the composite camera unit (100) captures physical flames or smoke, the precursor signal generated by the module can induce the system to prioritize and precisely monitor the area by adjusting the analysis weight of the AI ​​analysis unit (210) in advance.

[0323] The combination of the above-mentioned environmental acoustic collection unit (400) and the irregular acoustic gap analyzer (410) produces the following specific effects.

[0324] First, by capturing the acoustic gaps in ecosystems and facilities that respond to minute heat or vibrations before visible flames or smoke appear, it is possible to detect precursors several minutes faster than existing optical-based systems. This makes a decisive contribution to securing the golden time for initial suppression to prevent the spread of large-scale fires.

[0325] Second, even in extreme environments where the performance of the visible light and infrared sensors of the composite camera unit (100) is degraded, such as fog, heavy snow, heavy rain, or night, the sound data is less affected by physical impact. Therefore, even in situations where visual information is blocked, an independent detection path is provided, thereby raising the overall detection reliability of the system.

[0326] Third, under normal circumstances, all zones are monitored with equal weight, but only when a significant acoustic gap occurs in a specific zone is the analysis intensity of the AI ​​analysis unit (210) and the composite camera unit (100) concentrated on that point. Through this intelligent resource allocation, the system load is optimized, and at the same time, unnecessary waste of administrative power caused by false alarms is prevented by cross-verifying visual information and acoustic information.

[0328] This configuration departs from conventional methods that simply detect the occurrence of new signals and enables the implementation of an uninterrupted fire monitoring system even in environments where visual information is limited by quantifying the atypical pattern of background sound loss caused by fire precursors.

[0330] Although the present invention has been described above with reference to one embodiment, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims. Explanation of the symbols

[0331] Composite camera unit (100) Visible light imaging sensor (110) infrared sensor (120) scattered light detection sensor (130) Fire Integrated Control Server (200) AI Analysis Department (210) Field control unit (220) Integrated management interface (230) Big data learning module (240) Remote response instruction module (250) Administrator terminal (300)

Claims

Claim 1 The system includes a composite camera unit (100) installed in a fire monitoring target area to collect multispectral data related to fire in real time, and a fire integrated control server (200) to determine whether a fire has occurred by analyzing the data collected from the composite camera unit (100). The composite camera unit (100) includes a visible light image sensor (110) to capture images corresponding to the human visible light range to obtain visual information of the scene, an infrared sensor (120) to detect thermal radiation energy emitted from an object to collect temperature information, and a scattered light detection sensor (130) to detect the concentration and distribution of smoke using the principle of light scattering by smoke particles suspended in the air. The fire integrated control server (200) includes an AI analysis unit (210) to perform spatiotemporal alignment that aligns data with different resolutions and sampling periods, including visible light images, infrared thermal images, and scattered light sensor data, into the same time axis and coordinate system, and immediately generates an alarm and sends to the fire integrated control server (200) upon fire detection. The system includes a field control unit (220) for transmitting signals, an integrated management interface (230) for visualizing collected images and analysis results and providing them to a terminal (300), a remote response instruction module (240) for managing reporting to relevant agencies and remote control of on-site fire extinguishing equipment by combining the judgment result of the AI ​​analysis unit (210) and the final approval of the manager through the integrated management interface (230), a big data learning module (250) for accumulating and analyzing all detection data and judgment history occurring within the fire integrated control server (200), and a dynamic insurance rate calculation module (260) for combining fire monitoring data and financial indicators, wherein the AI ​​analysis unit (210) includes an intelligent time series analysis engine, and the intelligent time series analysis engine detects flame candidate regions in images acquired from a visible light image sensor (110), calculates flame fractal dimension values ​​by analyzing the irregular self-similarity of the contours of the regions, and analyzes thermal radiation energy detected through an infrared sensor (120).The method is characterized by calculating the temperature rise rate, which is a temperature change amount exceeding a specific threshold, in real time, deriving the smoke particle concentration change rate based on smoke particle density data measured by the scattered light detection sensor (130), and integrating and analyzing the extracted individual indicators through a multi-sensor fusion algorithm, while aligning brightness, temperature, and concentration data collected from different sensors to the same coordinate system and time axis to identify the correlation of complex physical quantities occurring at a specific point, wherein the dynamic insurance rate calculation module (260) quantifies the risk at the site based on the real-time fire occurrence probability transmitted from the AI ​​analysis unit (210) and the expected damage severity calculated through the data of the scattered light detection sensor (130) to derive a preliminary risk rate, calculates a compensation coefficient for the preliminary risk rate based on the accumulated accident-free operating time data stored in the big data learning module (250) to generate a final dynamic rate, and normalizes the real-time fire occurrence probability and the accumulated accident-free operating time to a value between 0 and 1 according to the system standard scale to synchronize the computation unit between the data, and regarding the accumulated accident-free operating time An AI-based multispectral image fusion type fire monitoring system characterized by applying a log-functional attenuation model to calculate a safety compensation coefficient by applying a weighted discount rate to the long-term accident-free maintenance period compared to the accident-free period at the beginning of operation, calculating a system reliability coefficient by combining the sensor alignment accuracy of the composite camera unit (100) and the false positive removal rate of the big data learning module (250), and performing a correction operation by subtracting the safety compensation coefficient from the preliminary risk rate and dividing the derived value by the system reliability coefficient, thereby controlling the final dynamic rate to decrease as the monitoring precision of the system increases. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete

Images