Fire perception system

Abstract

The present invention relates to an intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking, characterized by enabling life rescue even in fire situations where visibility or optical detection is blocked and access is difficult or nearly impossible due to location, promoting speed in determining priorities for life rescue under urgent circumstances, and actively blocking false fire detections through comprehensive discrimination of each sensing operation, thereby enhancing the reliability of fire detection and optimal response for each fire situation. The system comprises: a detection unit (100) that collects detection data regarding video and sound while detecting wavelengths and smoke emitted when flames are generated due to a combustion reaction; and an intelligence unit (200) that enables visual output by extracting objects from fire data and video based on previously learned fire situations and detection data for infrared, captured video, and smoke detection. It is composed of a control unit (300) that enables the output of detection data and fire data collected by a detection unit (100) and generates an alert based on the fire data of an intelligence unit (200), and is characterized by enabling accurate fire identification and rapid rescue of lives through three-dimensional complex detection based on multimodal sensor fusion even in a fire situation where visibility is obscured.

Description

Technology Field

[0001] The present invention advances the technical concept of an overall system related to fire detection, and more specifically, it relates to an intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking, which is characterized by enabling life rescue even in fire situations where visibility or optical detection is blocked and access is difficult or nearly impossible in terms of location, promoting speed in determining priorities for life rescue in urgent situations, and actively blocking false fire detections through comprehensive discrimination of each sensing operation, thereby enhancing the reliability of fire detection and optimal response for each fire situation. Background Technology

[0003] Modern buildings, such as communal living facilities, plant production facilities, and hazardous facilities, are densely packed in confined spaces, and this is all a result of modern living conditions.

[0005] In these buildings, various types of hazardous materials vulnerable to fire are stored and managed. Although it is fully foreseeable that massive damage could spread to the entire society in the event of a disaster due to factors such as the dense structure or characteristics of the building site, automatic fire detection systems have not been sufficiently installed for reasons of maintenance efficiency or economic feasibility.

[0007] Although automatic fire detection systems are mandatory for communal living facilities where a large number of people live together, the reality is that their functionality is not very good. Furthermore, even if fire detection systems are installed in facilities such as underground areas, there is a very high possibility that they will malfunction due to temperature, humidity, noise, etc.

[0009] In reality, in the event of a fire, if rescue personnel fail to locate the victims, it can lead to a situation where even their own lives are at risk. While early fire detection is crucial, there is a growing need for technological concepts capable of locating and rescuing lives even within an active fire.

[0011] For example, registered patent No. 10-2892072 discloses a technical concept regarding a 'fire early detection and suppression system and method'.

[0012] The technical concept relates to a system capable of detecting and suppressing a fire using a learned artificial intelligence algorithm. An early fire detection and suppression system according to one embodiment of the present invention includes an observation unit that generates observation data by observing a designated monitoring area, a memory and a processor in which a first artificial intelligence algorithm learned for classifying a monitoring target through first learning data and a second artificial intelligence algorithm learned for diagnosing an overheating state of a monitoring target through second learning data are stored. The memory has the advantage of being able to store program instructions that are executable by the processor, which classify a monitoring target using the first artificial intelligence algorithm using the observation data and diagnose an overheating state of the monitoring target classified by the first artificial intelligence algorithm using the second artificial intelligence algorithm using the observation data.

[0014] The aforementioned technological concept is focused on suppression rather than on saving lives, and therefore, further research on life-saving fire detection systems is needed in actual disaster environments. Prior art literature

[0016] Registered Patent Publication No. 10-2892072 (2025.11.28) The problem to be solved

[0017] The present invention was created to more actively resolve the aforementioned problems, and its main purpose is to provide an intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking that enables life rescue even in fire situations where visibility or optical detection is blocked and access is difficult or nearly impossible in terms of location, and promotes rapid determination of priority for life rescue in urgent situations.

[0019] In addition, another objective of the present invention is to provide an intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking, which is characterized by actively blocking false fire detections through comprehensive discrimination of each sensing operation, thereby enhancing the reliability of fire detection and optimal response for each fire situation. means of solving the problem

[0021] To achieve the above-mentioned problem, the intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking proposed by the present invention is as follows.

[0023] The present invention is characterized by being composed of: a detection unit (100) that collects detection data for video and sound while detecting wavelengths and smoke emitted when flames are generated due to a combustion reaction; an intelligence unit (200) that extracts objects from fire data and video based on previously learned detection data for fire situations, infrared, captured video, and smoke detection, and enables visual output; and a control unit (300) that enables the detection data and fire data collected by the detection unit (100) to be output and generates an alert based on the fire data of the intelligence unit (200), thereby enabling accurate fire identification and rapid rescue of lives through 3D composite detection based on multimodal sensor fusion even in fire situations where visibility is obscured.

[0025] The above detection unit (100) further comprises: a combustion sensor unit (110) that detects infrared rays emitted when a flame is generated due to a combustion reaction and collects flame detection data; a camera unit (120) composed of a plurality of cameras that captures a set area and is linked with the combustion sensor unit (110) to capture a fire occurrence area and collect video data; a smoke sensor unit (130) that detects smoke and collects smoke data; and an acoustic sensor unit (140) composed of a three-dimensional microphone that is composed of a plurality of microphones to precisely detect sound in the fire occurrence space and collects acoustic data, thereby actively blocking the spread of fire occurring within the space captured by the camera unit (120) and promoting rapid rescue of lives.

[0027] The above intelligence unit (200) further includes: a database unit (210) for managing prior data for prior learning about fire and for classifying and managing detection data collected by the detection unit (100); and a fire intelligence unit (220) for generating fire data based on infrared, image, smoke, and sound of the prior data managed in the database unit (210) and the collected detection data, and for extracting objects from the image detection data.

[0029] The above database unit (210) further includes a pre-learning unit (211) for classifying and managing pre-learning data for fire; and a detection classification unit (212) for classifying and managing detection data collected by the detection unit (100) into infrared, captured images, smoke, and sound.

[0031] The above detection classification unit (212) further includes: a flame detection classification unit (212a) that stores and manages flame detection data collected through the combustion sensor unit (110); an image classification unit (212b) that stores and manages image data of a set zone and a fire occurrence zone collected through the camera unit (120); a smoke classification unit (212c) that stores and manages smoke data collected through the smoke sensor unit (120); and an acoustic classification unit (212d) that stores and manages acoustic data collected through the acoustic sensor unit (130).

[0033] The above fire intelligence unit (220) comprises: an infrared detection unit (221) capable of generating flame detection data by determining the infrared generation location based on prior data and detection data from infrared detection; an image detection unit (222) capable of generating image detection data that visually displays the infrared generation location by extracting an object from the detection data of a camera image capturing the infrared generation location in conjunction with the infrared detection unit (221); a smoke fire detection unit (223) capable of generating smoke detection data based on detection data for smoke; an infrared detection unit (221) and an image detection unit (222); an acoustic detection unit (224) positioned in a space corresponding to the smoke fire detection unit (223) to classify acoustic data regarding collected noise and generate acoustic detection data in which the generation location of the acoustic data can be confirmed; an infrared detection unit (221) and an image detection unit (222); and a smoke fire detection unit (223). It further includes a fire data generation unit (225) that generates fire data based on each discrimination data of the sound discrimination unit (224).

[0035] The above-mentioned sound discrimination unit (224) further includes a sound classification unit (224a) that classifies sound data regarding noise collected through learning of non-verbal sound or verbal sound into non-verbal or verbal sound data, and a sound source location tracking unit (224b) that enables tracking the location of non-verbal or verbal sound data.

[0037] The above fire data generation unit (225) further includes: an operation determination unit (225a) that enables the generation of fire operation data based on whether two or more infrared, image, and smoke detection operations of the detection unit (100) occur simultaneously or sequentially; and a fire determination unit (225b) that generates fire data regarding whether a fire has occurred based on the fire operation data of the operation determination unit (225a) and the determination data of the infrared determination unit (221), image determination unit (222), smoke fire determination unit (223), and sound determination unit (224).

[0039] The present invention may further include the following configurations.

[0041] The above control unit (300) further includes: a screen output unit (310) that enables real-time monitoring by outputting detection data collected by the detection unit (100) and fire data from the intelligence unit (200); and an alert generation unit (320) that generates an alert based on the fire data from the intelligence unit (200).

[0043] The above detection unit (100) further includes a laser unit (150) that emits a laser in conjunction with a camera unit (120), and is characterized by allowing a person in the space where the fire occurred to clearly recognize the risk of fire by emitting a laser according to the risk level on the fire data of the fire detection unit (225b).

[0045] The above laser unit (150) further includes a laser emitting unit (151) that emits light according to the risk level determined by fire data; and a shape determining unit (152) that gives a shape to the light emitted from the laser emitting unit (151). Effects of the invention

[0047] According to the present invention, which is configured as described above, it is possible to rescue lives even in fire situations where visibility or optical detection is blocked and access is difficult or nearly impossible due to location, and it is possible to ensure speed in determining priorities for life rescue in urgent situations.

[0049] In addition, the present invention can enhance the reliability of fire detection and optimal response for each fire situation by actively blocking false fire detections through the comprehensive determination of each sensing operation. Brief explanation of the drawing

[0051] FIG. 1 is a schematic diagram of the fire response system of the present invention. FIG. 2 is a block diagram of the fire response system of the present invention. FIG. 3 is a detailed block diagram of the fire response system of the present invention. FIG. 4 is another embodiment of the fire response system of the present invention. Specific details for implementing the invention

[0052] First, the advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Furthermore, throughout the entire specification, the same reference numerals refer to the same components.

[0054] It should be noted that the present invention relates to an intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking, which is characterized by enabling life rescue even in fire situations where visibility or optical detection is blocked and access is difficult or nearly impossible in terms of location, promoting speed in determining priorities for life rescue in urgent situations, and actively blocking false fire detections through comprehensive discrimination of each sensing operation, thereby enhancing the reliability of fire detection and optimal response for each fire situation.

[0056] Throughout the specification, when it is stated that a part is "connected" to another part, this includes not only cases where it is "directly connected," but also cases where it is "electrically connected" with other components interposed between them. Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0057] In addition, the present invention lists systemic configurations, wherein the name of '...part' may be composed of hardware or a combination of hardware and software, and may also mean a device including a functional expression.

[0059] Hereinafter, the structure and effects of the present invention will be described collectively.

[0061] As illustrated in FIG. 1, the present invention comprises: a detection unit (100) that collects detection data for video and sound while detecting wavelengths and smoke emitted when flames are generated due to a combustion reaction; an intelligence unit (200) that generates fire data and extracts objects from video based on detection data for a previously learned fire situation, infrared, captured video, and smoke detection, and enables visual output; and a control unit (300) that enables the output of detection data and fire data collected by the detection unit (100) and generates notifications based on the fire data of the intelligence unit (200). The invention is characterized by enabling accurate fire identification and rapid rescue of lives through three-dimensional composite detection based on multimodal sensor fusion, even in a fire situation where visibility is obscured.

[0063] Traditionally, fire detection has relied on flames or smoke. This limited detection method is unsuitable for the current dense structure of buildings and has posed significant problems in the event of a fire in facilities located underground. Its narrow function, which is limited to simple detection followed by a fire alarm, is woefully inadequate for detecting unexpected fires and ultimately saving lives in the event of a fire.

[0064] To overcome this, the present invention is configured to detect flame generation through infrared sensors, detect smoke by inhaling smoke, and initiate video recording of the monitored target in conjunction with the occurrence of flames and smoke when power is constantly on or power supply is cut off after a certain period of time, thereby ensuring accuracy in fire detection and enabling initial suppression, which can lead to the early cessation of the fire. Furthermore, even if visibility is completely obstructed due to smoke or other reasons during the spread of the fire, sound can be detected through a micro array microphone, and by distinguishing rescue noises and tracking locations, a basis for the speed of rescue and rescue priority can be established.

[0066] Specifically, the detection unit (100) may be configured to detect flames based on wavelengths, such as an infrared sensor, detect smoke generated by the combustion of objects, etc., so as to detect whether a fire has occurred in a monitored area (including all spatial meanings of zones and places), a camera installed in the monitored area and capturing a specific area, and a microphone that collects sound in the area. Each collected infrared, smoke, camera, and sound is referred to as detection data.

[0067] The above-mentioned intelligence unit (200) can utilize the type of combustible material, the ignition point, the fire spread speed and extinguishing speed in the event of a fire occurring in a similar monitoring target area as prior learning data to learn about fire situations and generate fire data based on each detection data. At this time, the fire data can be configured to determine and calculate the fire spread speed when a fire occurs in a monitoring target area through the type of combustible material, the presence of combustible material in the monitoring target area, the location of the flames, and the fire spread speed in the event of a fire occurring in a similar monitoring target area (similar building). To this end, an artificial intelligence machine learning algorithm may be employed and configured to enable continuous learning, thereby enabling optimal response to unpredictable fire situations. Additionally, the flame occurrence point in the video detection data may be indexed (highlighted with a box, color, etc.) so that it can be output and verified by the control unit.

[0068] The above control unit (300) can be implemented as a terminal or PC device that can be operated by a user or administrator, and may be interconnected with the detection unit (100) and the intelligence unit (200). The control unit can manage the prior data of the intelligence unit (200) through a separate input, and is configured to output video detection data from the detection unit (100) or video detection data that has passed through the intelligence unit (200), and may further include an output means (monitor), etc. At this time, based on the sound detection data, the location tracking value where the life-saving noise occurred can be configured to be merged with the video detection data and output.

[0069] The above-mentioned sensing unit (100) and intelligence unit (200) may be connected via wired or wireless communication, and the control unit (300) and intelligence unit (200) may be connected via a wireless network, mobile communication network, etc.

[0070] The configuration of the present invention described above is characterized by ensuring the accuracy of fire detection through the continuous operation of the detection unit (100), thereby actively preventing situations where social costs and meaningless fear are caused by misjudgment of fire, enabling life rescue even in situations where visibility or optical detection is blocked and access is difficult or nearly impossible due to location, and enhancing reliability by providing speed and grounds for determining priority for life rescue in urgent situations.

[0072] The present invention further includes the following configuration to achieve the purpose and effects of the present invention as described above.

[0074] As illustrated in FIGS. 2 and 3, the detection unit (100) further comprises: a combustion sensor unit (110) that detects infrared rays emitted when a flame is generated due to a combustion reaction and collects flame detection data; a camera unit (120) composed of a plurality of cameras that captures a set area and is linked with the combustion sensor unit (110) to capture a fire occurrence area and collect video data; a smoke sensor unit (130) that detects smoke and collects smoke data; and an acoustic sensor unit (140) composed of a three-dimensional microphone that is composed of a plurality of microphones to precisely detect sound in the fire occurrence space and collects acoustic data, thereby actively blocking the spread of fire occurring within the space captured by the camera unit (120) and promoting rapid rescue of lives.

[0075] The above combustion sensor unit (110) can be configured to detect flames generated in a monitored area based on infrared rays.

[0076] The above camera unit (120) may refer to a camera that is positioned in a detection target area and can capture images in conjunction with motion or the operation of the combustion sensor unit (110).

[0077] The above smoke sensor unit (130) can detect smoke generated by the combustion of combustible material and convert it into smoke data.

[0078] The above acoustic sensor unit (140) may be configured as a microphone array that arranges multiple microphones in a specific pattern to perform acoustic signal processing that is impossible with a single microphone, namely, focusing on listening to sounds from a specific direction, eliminating noise and reverberation, and identifying and tracking the location of the sound source. The acoustic sensor unit may be an important factor in determining whether a fire has occurred in the detection target area, and, for example, can detect explosions of combustible materials or damage to surrounding objects due to fire through sound, and can track the location of people through sound when the fire spreads.

[0080] The above intelligence unit (200) further includes: a database unit (210) for managing prior data for prior learning about fire and for classifying and managing detection data collected by the detection unit (100); and a fire intelligence unit (220) for generating fire data and extracting objects from image detection data based on infrared, image, smoke, and sound of the prior data managed in the database unit (210) and the collected detection data.

[0081] The above database unit (210) may be composed of a database and further includes the operation of a DBMS (Database Management System) to manage a set of databases, etc.

[0082] Specifically, the database unit (210) further includes a pre-learning unit (211) for classifying and managing pre-learning data for fire; and a detection classification unit (212) for classifying and managing detection data collected by the detection unit (100) into infrared, captured images, smoke, and sound.

[0083] The above-mentioned prior learning unit (211) can be configured to be able to learn prior learning sounds related to fire, such as the type of combustible material, ignition point, fire spread speed and extinguishing speed when a fire occurs in a similar monitoring target area, sound of breaking objects, voice of a person, explosion sound, fire alarm, and sprinkler operation sound, classified as non-verbal or verbal, as described above. To implement this, a deep learning-based Sound Event Detection model may be utilized.

[0084] The above detection classification unit (212) further includes: a flame detection classification unit (212a) that stores and manages flame detection data collected through the combustion sensor unit (110); an image classification unit (212b) that stores and manages image data of a set zone and a fire occurrence zone collected through the camera unit (120); a smoke classification unit (212c) that stores and manages smoke data collected through the smoke sensor unit (120); and an acoustic classification unit (212d) that stores and manages acoustic data collected through the acoustic sensor unit (130).

[0086] The above fire intelligence unit (220) comprises: an infrared detection unit (221) capable of generating flame detection data by determining the infrared generation location based on detection data from infrared detection; an image detection unit (222) connected to the infrared detection unit (221) capable of generating image detection data that extracts objects from the detection data of a camera image capturing the infrared generation location, extracts objects within the captured image, and visually displays the infrared generation location; a smoke fire detection unit (223) capable of generating smoke detection data based on detection data for smoke; an infrared detection unit (221) and an image detection unit (222); an acoustic detection unit (224) positioned in a space corresponding to the smoke fire detection unit (223) capable of classifying acoustic data regarding collected noise and generating acoustic detection data in which the generation location of the acoustic data can be confirmed; an infrared detection unit (221) and an image detection unit (222); and a smoke fire detection unit (223). It further includes a fire data generation unit (225) that generates fire data based on each discrimination data of the sound discrimination unit (224).

[0088] The infrared detection unit (221) can generate flame detection data by determining the infrared generation location based on detection data from infrared detection. Since several units may be placed in the detection target area according to the detection range of the combustion sensor unit (110), it is necessary to confirm the location of a specific combustion sensor unit (110). To this end, a unique identification number may be set for each combustion sensor unit (110). For example, a number of combustion sensor units (110) may be provided, and the placement locations may be arranged such that they are spaced apart from each other by a certain distance according to the detection range of each combustion sensor unit (110). Each combustion sensor unit (110) may be set with a number or a number composed of a number and a letter to specify its location.

[0089] The camera unit (120) can start taking pictures at the location of the combustion sensor unit (110) that detects infrared rays emitted from the flame. At this time, the camera unit (120) may also be configured in multiple units and may be positioned to enable taking pictures of the area where each combustion sensor unit (110) is placed.

[0091] The above image discrimination unit (222) is configured to analyze the captured image of the camera unit (120), and more specifically, it is configured to extract objects from the detection data of the camera image that captures the infrared generation location in conjunction with the infrared discrimination unit (221), which recognizes the area where the flame is detected and the combustible material in the surrounding area as objects and classifies them, and the area where the flame is generated can be configured to generate image discrimination data in which shapes such as squares and triangles are visually displayed.

[0092] Additionally, the image identification data may be configured to calculate the fire spread speed based on the fire spread speed in a fire that occurs in a combustible material of the preceding data or a similar monitoring target area (a building or space separately designated by the user as a building manufactured in the same form as the monitoring target area), taking into account the existence and type of an object selected as an object of the said monitoring area. This allows the fire spread speed to be classified as exceeding the golden time when there are three or more combustible materials, starting from the 7-minute golden time of the fire, thereby classifying it as a high-risk fire spread speed. In other words, it may be configured to simulate a fire situation based on the combustible material or the situation of the monitoring target area in the image identification data, and this operation may be performed in conjunction with the fire data generation unit (225).

[0094] The above smoke fire detection unit (223) detects smoke and generates smoke detection data, wherein the smoke detection data is data that determines whether smoke formed by a combustion reaction continues to be generated and the amount of smoke, and can be configured to compensate for malfunctions of the sensor or camera depending on whether such smoke detection data is generated.

[0096] The above-mentioned acoustic discrimination unit (224) is placed in a space corresponding to the infrared discrimination unit (221), the image discrimination unit (222); and the smoke / fire discrimination unit (223) and classifies acoustic data regarding collected noise into non-verbal and verbal sounds. This non-verbal and verbal classification can be based on learned prior data and can utilize a deep learning-based Sound Event Detection model.

[0097] The sound collected in the monitored area is used to calculate the location where the sound originated in 3D coordinates using Acoustic Source Localization technology. By analyzing the time difference in which the sound reaches each microphone of the microphone array of the acoustic sensor unit (140), the direction and distance of the sound source can be estimated, which means that sound identification data is generated. The estimation of the direction and distance of the sound source can also be used as supporting data for cross-verification with the location of fire occurrence on the image identification data of the image identification unit (223).

[0098] To this end, the above-mentioned sound discrimination unit (224) further includes: a sound classification unit (224a) that classifies sound data regarding noise collected through learning of non-verbal sound or verbal sound into non-verbal or verbal sound data; and a sound source location tracking unit (224b) that enables tracking the location of non-verbal or verbal sound data.

[0100] The above fire data generation unit (225) can generate fire data based on flame discrimination data in which the location of each combustion sensor unit (110) is determined by flame detection, image discrimination data in which combustible materials in the area where flames are detected and surrounding areas are recognized as objects and the area where flames are generated is visually displayed as a shape such as a square or a triangle, smoke discrimination data in which the amount of smoke generated by the combustion reaction is determined and whether smoke continues to be generated, and sound discrimination data in which the coordinates of the location of the sound source are estimated in addition to sound data classified into non-verbal and verbal based on deep learning.

[0101] The fire data can be configured to determine whether a fire has occurred based on each determination data. Specifically, it includes determining whether a fire has occurred based on whether each determination data is generated, that is, whether the combustion sensor unit (110), camera unit (120); smoke sensor unit (130); and sound sensor unit (140) are operated. For example, if the combustion sensor unit (110) or camera unit (120) is operated but the smoke sensor unit (130) is not operated, it can ultimately be determined that no fire has occurred. It can be configured to determine that a fire has occurred only when all detections, such as combustion, camera, and smoke, are operated.

[0102] In addition, the system can be configured to calculate the re-spread rate based on the presence or absence of combustible materials in objects extracted from video identification data. This allows for determining that the fire spread rate is high-risk by setting the fire spread rate to exceed the golden time of 7 minutes as the threshold, if there are three or more combustible materials or if the combustible materials burn very quickly. Furthermore, the risk of fire spread can be calculated based on the continuity of detection in flame identification data, the continuity of smoke detection in smoke identification data, and non-verbal and verbal sounds in sound identification data, and these can be integrated into a single fire data set. The risk can be calculated by classifying it into grades 1, 2, 3, 4, 5, etc., or it can be configured to focus solely on non-verbal and verbal sounds to classify a fire as high-risk when high-pitched sounds occur or when human life is present in the monitored area.

[0103] In detail, the fire data generation unit (225) further includes: an operation determination unit (225a) that enables the generation of fire operation data based on whether two or more infrared, image, and smoke detection operations of the detection unit (100) occur simultaneously or sequentially; a fire determination unit (225b) that generates fire data regarding whether a fire has occurred based on the fire operation data, infrared determination unit (221), image determination unit (222); smoke fire determination unit (223); and sound determination unit (224).

[0105] Each operation and detection history, fire data, and video data as described above can be output to and practically managed by the control unit (300), and for this purpose, the control unit (300) can be configured to be operable using a PC, terminal, mobile phone, etc.

[0107] To this end, the control unit (300) further includes: a screen output unit (310) that enables real-time monitoring by outputting detection data collected by the detection unit (100) and fire data from the intelligence unit (200); and an alert generation unit (320) that generates an alert based on the fire data from the intelligence unit (200).

[0108] The above screen output unit (310) can be configured to output all operation and detection details of the combustion sensor unit (110); camera unit (120); smoke sensor unit (130); and sound sensor unit (140), and can be configured to output flames, video, smoke, and sound on one screen through a divided format of at least one section. Of course, real-time monitoring can be enabled by optionally outputting video in real time.

[0109] The above notification generation unit (320) can generate a notification using sound, light, etc., according to the risk level of fire operation data and fire data, and can be configured to output such notification to the screen output unit (310).

[0111] The present invention may further include the following configurations.

[0113] The above detection unit (100) further includes a laser unit (140) that emits a laser in conjunction with a camera unit (120), and is characterized by allowing a person in the space where the fire occurred to clearly recognize the risk of fire by emitting a laser according to the risk level determined by the fire data of the fire determination unit (225b).

[0114] The above laser unit (150) further includes a laser emitting unit (151) that emits light according to the risk level determined by fire data; and a shape determining unit (152) that gives a shape to the light emitted from the laser emitting unit (151).

[0115] The above laser emitting unit (151) is configured to amplify light generated in a laser medium from a laser device, i.e., a laser device, and produce and emit strong, concentrated light in a desired direction.

[0116] When a simple straight laser is emitted, it is practically difficult to identify in a fire situation. To this end, the shape determining part (152) can have a specific shape when the laser is emitted and irradiated onto the ground or wall.

[0117] The shape determining part (152) above can be implemented in a pre-set form and can be configured in the form of a direction table corresponding to a contrast path of the monitored area.

[0118] This laser emitting unit (151) can be operated only in high-risk fires where human life is present in the monitored area, thereby enabling efficient maintenance and management of the system operation.

[0120] The present invention, as described above, has the advantage of enabling life rescue even in situations where visibility or optical detection is blocked and access is difficult or nearly impossible due to location, under fire detection relying solely on visual and optical detection, and can facilitate rapid determination of priorities for life rescue in urgent situations.

[0121] In addition, there is another advantage in that it enables optimal response for each fire situation and actively blocks false fire detections through comprehensive identification of each sensing operation, thereby enhancing the reliability of fire detection.

[0123] To explain the above functions, effects, configurations, and operations, reference has been made to an exemplary embodiment illustrated in the drawings, but this is merely illustrative and should be made clear to those skilled in the art that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be interpreted by the appended claims, and all technical ideas within an equivalent scope should be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0125] 100: Detection unit 110: Combustion sensor unit 120: Camera unit 130: Smoke sensor unit 140 : Acoustic sensor unit 200 : Intelligence unit 210: Database Section 211: Prerequisites Section 212: Detection Classification Unit 220: Fire Intelligence Unit 221 : Infrared Discrimination Unit 222 : Image Discrimination Unit 223 : Smoke / Fire Identification Unit 224 : Sound Identification Unit 225 : Fire Data Generation Unit 300 : Control Department

Claims

Claim 1 A fire response system that aims for accurate fire identification and rapid rescue of lives through 3D composite detection based on multimodal sensor fusion even in a fire situation where visibility is obscured, comprising: a detection unit (100) that collects detection data for video and sound while detecting wavelengths and smoke emitted when flames are generated due to a combustion reaction; an intelligence unit (200) that extracts objects from fire data and video based on previously learned fire situations, infrared, captured video, and smoke detection detection data, and enables visual output; and a control unit (300) that enables the output of detection data and fire data collected by the detection unit (100) and generates notifications based on the fire data of the intelligence unit (200); wherein the detection unit (100) comprises: a combustion sensor unit (110) that collects flame detection data by detecting infrared rays emitted when flames are generated due to a combustion reaction; a camera unit (120) composed of a plurality of cameras that captures a set area and captures a fire occurrence area and collects video data in conjunction with the combustion sensor unit; and smoke detection and smoke data collection The system comprises a smoke sensor unit (130); an acoustic sensor unit (140) composed of a three-dimensional microphone that collects acoustic data by assembling a plurality of microphones to precisely detect sound in the space where a fire occurs; and a laser unit (150) linked to a camera unit, thereby blocking the spread of fire occurring within the space captured by the camera unit and promoting rapid rescue of lives. The intelligence unit (200) includes a database unit (210) that manages prior data for prior learning about fire and classifies and manages detection data collected by the detection unit; and a fire intelligence unit (220) that generates fire data based on infrared, video, smoke, and sound of the prior data managed by the database unit and the collected detection data, and extracts objects from the video detection data.The fire intelligence unit (220) comprises: an infrared detection unit (221) capable of generating flame detection data by determining the infrared generation location based on detection data from infrared detection; an image detection unit (222) capable of generating image detection data that visually displays the infrared generation location by extracting an object from the detection data of a camera image capturing the infrared generation location in conjunction with the infrared detection unit; a smoke fire detection unit (223) capable of generating smoke detection data based on detection data for smoke; an acoustic detection unit (224) capable of classifying acoustic data regarding noise collected by being placed in a space corresponding to the infrared detection unit, the image detection unit, and the smoke fire detection unit, and generating acoustic detection data in which the generation location of the acoustic data can be confirmed; and a fire data generation unit (225) capable of generating fire data based on the detection data of each of the infrared detection unit, the image detection unit, the smoke fire detection unit, and the acoustic detection unit.The system further includes an acoustic discrimination unit (224) comprising an acoustic classification unit (224a) that classifies acoustic data regarding noise collected through learning of non-verbal or verbal acoustics into non-verbal or verbal acoustic data, and a sound source location tracking unit (224b) that enables tracking the location of non-verbal or verbal acoustic data, and is configured to calculate the location of the occurrence of life-saving noise in 3D coordinates based on a Microphone Array, Acoustic Source Localization, and TDOA (Time Difference of Arrival); and the laser unit (150) comprising a laser emission unit (151) that emits a laser according to a fire risk calculated based on fire data, and a shape determination unit (152) that projects the laser emitted from the laser emission unit into a pre-set shape to visually display an evacuation path, thereby operating in a high-risk fire situation where the presence of human life is determined to display a guide shape indicating the evacuation direction on the floor or wall of the monitored area, thereby inducing rapid evacuation of human life within the fire-affected space. Intelligent fire response system using deep learning-based multimodal sensor fusion and sound source localization. Claim 2 delete Claim 3 delete Claim 4 An intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking, wherein the database unit (210) further comprises: a pre-learning unit (211) for classifying and managing pre-learning data for pre-learning about fire; and a detection classification unit (212) for classifying and managing detection data collected by the detection unit (100) into infrared, captured image, smoke, and sound. Claim 5 An intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking, wherein the detection classification unit (212) further comprises: a flame detection classification unit (212a) that stores and manages flame detection data collected through a combustion sensor unit (110); an image classification unit (212b) that stores and manages image data of a set zone and a fire occurrence zone collected through a camera unit (120); a smoke classification unit (212c) that stores and manages smoke data collected through a smoke sensor unit (130); and an acoustic classification unit (212d) that stores and manages acoustic data collected through an acoustic sensor unit (140). Claim 6 delete Claim 7 delete Claim 8 In claim 1, the fire data generation unit (225) further comprises: an operation determination unit (225a) capable of generating fire operation data based on whether two or more infrared, image, and smoke detection operations of the detection unit (100) are performed simultaneously or sequentially; and a fire determination unit (225b) generating fire data regarding whether a fire has occurred based on fire operation data and determination data of each of the infrared determination unit (221), image determination unit (222), smoke fire determination unit (223), and sound determination unit (224). This describes an intelligent fire response system using deep learning-based multimodal sensor fusion and sound source location tracking.

Images