Fire detection device
The fire detection device improves accuracy and robustness by using small-region and label-region calculation units with time-series data and machine learning discrimination to identify flame regions, addressing false alarms and environmental variations.
Patent Information
- Application Number
- JP2023054473
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-03-30
- Publication Date
- 2026-11-19
- Estimated Expiration
- 2043-03-30
AI Technical Summary
Conventional fire detection methods using feature values in small regions are prone to false alarms when smoke generation areas are obscured, and the feature quantities obtained are insufficient for accurate fire detection, especially in various monitoring environments, and the optical system contamination can affect detection performance.
A fire detection device that performs calculation processing on input images from a surveillance camera, utilizing small-region and label-region calculation units, time-series data storage, and machine learning discrimination to identify flame regions, enhancing detection accuracy and robustness across diverse environments.
The device achieves highly accurate and robust fire detection capable of handling various monitoring environments, including partial occlusions and optical system contamination, by integrating spatial and time-varying features through machine learning, reducing false alarms.
Smart Images

Figure 0007954112000001 
Figure 0007954112000002 
Figure 0007954112000003
Abstract
Description
Technical Field
[0001] The present disclosure relates to a fire detection device that identifies a flame region by performing arithmetic processing on an input image captured by a surveillance camera.
Background Art
[0002] There is a conventional device that identifies a flame region by detecting smoke generated in a monitoring target area using a feature value focused on the flow characteristic unique to smoke (see, for example, Patent Document 1). In Patent Document 1, feature values are calculated in units of individual divided regions obtained by dividing the input image into arbitrary sections, flowing smoke is detected in consideration of the connection state of the feature values in the division units, and the flame region is determined.
[0003] Here, the divided regions correspond to small regions grouped in units of several pixels, and examples of the small regions include regions such as 3×3 pixels and 5×5 pixels. In Patent Document 1, feature values are calculated in consideration of spatially adjacent pixels. In addition, for input images captured at fixed sampling intervals, feature values are calculated in consideration of the adjacency relationship between small regions arranged in time series.
[0004] The conventional device according to Patent Document 1 detects smoke flowing in the monitoring target area based on the feature values calculated in this manner, and aims to improve smoke detection performance.
Prior Art Documents
Patent Documents
[0005]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0006] However, the conventional technique has the following problems. In Patent Document 1, if the area where smoke is being generated is partially obscured, the feature quantities in the small area change, and if this technology is applied to the detection of flame areas, it is possible that it will cause false alarms. Furthermore, the feature quantities obtained in small areas are similar to looking at a landscape through a small window, and the number of features obtained is small. In order to improve the fire detection accuracy to a certain level or higher, there are challenges in that, in addition to this method, it is necessary to find effective feature quantities for distinguishing between fire areas and non-fire areas for a specific phenomenon and add an appropriate alternative method.
[0007] Furthermore, in order to detect flame regions with high accuracy in various monitoring environments, it is important to consider not only the fluidity of smoke but also various features that are effective for fire detection according to the monitoring purpose, thereby improving detection accuracy while suppressing false alarms.
[0008] Furthermore, depending on the monitoring environment, the optical system of the surveillance camera may become contaminated, and the input image captured by the surveillance camera may be affected by this contamination, potentially negatively impacting the fire detection performance.
[0009] Therefore, there is a strong demand for a device that can improve fire detection accuracy while suppressing false alarms in various installation environments, depending on the monitoring purpose.
[0010] This disclosure is made to solve the above-mentioned problems and aims to provide a fire detection device with robustness that can be adapted to various monitoring environments. [Means for solving the problem]
[0011] The fire detection device according to this disclosure is a fire detection device that identifies a flame region by performing calculation processing on an input image captured by a surveillance camera, and comprises a small-region calculation processing unit that performs calculation processing on small-region units, which are units of several pixels that make up the input image, and a label region calculation processing unit that performs calculation processing on each label region obtained by performing labeling processing to integrate adjacent small regions that satisfy predetermined conditions based on the processing results of the small-region unit by the small-region calculation processing unit, wherein the small-region calculation processing unit has a time-series storage area that stores time-series data relating to each of a plurality of feature quantities calculated by the calculation processing on the small-region unit in order to identify the flame region, and The system further includes a flame region identification unit that calculates spatial features or time-varying features in each of the small regions, stores the calculation results for each feature over a predetermined period from the past to the present as time-series data in a time-series memory area, extracts the label regions obtained by performing labeling processing based on the time-series data of multiple features stored in the time-series memory area as fire determination processing areas, calculates statistical quantities of the time-series data related to the spatial features or time-varying features in each of the extracted fire determination processing areas, and identifies the flame region by performing machine learning discrimination processing based on the multiple statistical quantities calculated by the label region identification unit. [Effects of the Invention]
[0012] According to this disclosure, a fire detection device with robustness capable of handling various monitoring environments can be obtained. [Brief explanation of the drawing]
[0013] [Figure 1] This is a functional block diagram of the fire detection device in Embodiment 1 of the present disclosure. [Figure 2] This is a flowchart showing a series of processes performed for each image sample in the fire detection device according to Embodiment 1 of this disclosure. [Figure 3] This is an explanatory diagram relating to the labeling process by the label area calculation processing unit according to Embodiment 1 of the present disclosure. [Figure 4] This is an explanatory diagram showing the flow of a series of processes in a fire detection device according to Embodiment 2 of this disclosure. [Figure 5] This is an explanatory diagram showing the flow of a series of processes in a fire detection device using the first machine learning discriminant according to Embodiment 3 of this disclosure. [Figure 6] This is an explanatory diagram showing the flow of a series of processes in a fire detection device using a second machine learning discriminant according to Embodiment 3 of this disclosure. [Figure 7] This is an explanatory diagram showing the flow of a series of processes in a fire detection device that uses the discriminator according to Embodiment 3 of this disclosure in combination with the first machine learning discriminator and the second machine learning discriminator. [Modes for carrying out the invention]
[0014] Hereinafter, preferred embodiments of the fire detection device of this disclosure will be described with reference to the drawings. This disclosure describes in detail, in four embodiments, how each embodiment of the fire detection device possesses robustness that allows it to adapt to various monitoring environments.
[0015] In Embodiment 1, candidate regions for identifying flame regions are extracted by performing calculation processing in stages at the pixel level, sub-region level, and label region level. The technical feature is that the flame region can be identified based on the feature quantities of the finally extracted candidate regions. As a result, a fire detection device can be realized that is highly accurate, resistant to partial shielding, and robust enough to handle various monitoring environments.
[0016] In Embodiment 2, a technical feature is that a plurality of small classifiers configured by individually dividing the conditions required for fire determination are used to identify whether each label region is a fire region based on a combination of one or more statistics calculated for each label region, and further, a discriminator is used to identify a flame region from a combination of the identification results of the plurality of small classifiers. As a result, even with small classifiers that have low accuracy when used alone, a flame region can be identified with high accuracy from the combination of the identification results of the plurality of small classifiers according to the purpose of fire detection, and a fire detection apparatus with robustness that can adapt to various monitoring environments can be realized.
[0017] In Embodiment 3, a technical feature is that instead of the discriminator constructed by the designer in Embodiment 2, a machine learning discriminator that identifies a flame region from a combination of the identification results of a plurality of small classifiers, or a machine learning discriminator that identifies a flame region from a combination of one or more statistics calculated for each label region, is provided as a flame region identification unit that performs machine learning discrimination processing. Further, in the third embodiment, a technical feature is that in addition to the discriminator constructed by the designer in Embodiment 2, a configuration using a machine learning discriminator in combination is provided as a flame region identification unit that performs machine learning discrimination processing. As a result, by using discriminators with different characteristics, a flame region can be identified with high accuracy according to the purpose of fire detection, and a fire detection apparatus with robustness that can adapt to various monitoring environments can be realized.
[0018] In Embodiment 4, a technical feature is that focusing on the amount of infrared energy in an infrared image captured by an infrared thermography, and assuming a state where the optical system of the infrared thermography is contaminated and the transmittance is degraded, a configuration for determining a temperature threshold for identifying a flame region is provided. As a result, even if the contamination state changes, continuous fire monitoring can be performed without sensitivity compensation, and a fire detection apparatus with robustness that can adapt to various monitoring environments can be realized.
[0019] Embodiment 1. Figure 1 is a functional block diagram of a fire detection device according to Embodiment 1 of the present disclosure. The fire detection device according to Embodiment 1 has a function of identifying a flame region by performing arithmetic processing on an input image captured by a monitoring camera 1, and is configured to include an image memory 10 and an arithmetic processing unit 20.
[0020] The image memory 10 is configured as an image memory for a plurality of frames so that input images captured by the monitoring camera 1 at each predetermined sampling cycle can be stored as time-series images for a certain past period. An image for one frame is composed of a plurality of pixels.
[0021] Further, the arithmetic processing unit 20 is configured to include a pixel arithmetic processing unit 21, a small region arithmetic processing unit 22, and a label region arithmetic processing unit 23, and extracts candidate regions for identifying the flame region by performing three stages of arithmetic processing.
[0022] That is, the fire detector according to Embodiment 1 of the present disclosure has the functions as shown in Figure 1, so as to narrow down the fire region step by step, and can calculate the statistic in the narrowed-down fire region, thereby enabling highly accurate and robust fire detection that is resistant to partial occlusion.
[0023] Figure 2 is a flowchart showing a series of processes executed for each image sampling in the fire detection device according to Embodiment 1 of the present disclosure. In step S210, the fire detection device acquires an input image captured by the monitoring camera 1, and stores the input image in the image memory 10.
[0024] In the following description, an infrared camera such as an infrared thermography camera is used as the monitoring camera 1, and a case where values corresponding to temperature information are obtained as data of each pixel will be described as a specific example. In this case, for example, the luminance value 0 to 255 of each pixel corresponds to 0 to 255°C.
[0025] After acquiring an input image from the surveillance camera 1, the fire detection device sequentially performs calculation processing at the pixel level in step S220, calculation processing at the small area level in step S230, and calculation processing at the label area level in step S240.
[0026] The pixel-level calculation process in step S220 is comprised of steps S221 to S223 and is executed by the pixel calculation processing unit 21. In step S221, the pixel calculation processing unit 21 extracts high-temperature regions for each pixel of the input image using a pre-set temperature threshold for extracting high-temperature regions. For example, a temperature of 200°C is set as the temperature threshold, which is capable of detecting flames and excluding normal ambient temperatures.
[0027] Next, in step S222, the pixel processing unit 21 performs an intrusion region extraction process to determine whether there are any newly intruded objects in the image. Specifically, the pixel processing unit 21 creates a probability density distribution of background pixels from multiple input images in a normal state without fire.
[0028] The pixel processing unit 21 then determines that a new object has entered the image if the value of the target pixel in the input image acquired during fire monitoring corresponds to a value with a low frequency of appearance in the probability density distribution. By performing this extraction of intrusion regions, the area where the fire occurred can be identified as an intrusion region.
[0029] Furthermore, when creating the probability density distribution of background pixels, for example, known kernel density estimation methods can be applied, and a detailed explanation will be omitted.
[0030] Next, in step S223, the pixel arithmetic processing unit 21 performs frame difference processing to calculate the difference for each pixel based on the input image acquired during the previous sampling and the input image acquired during the current sampling. After that, the process proceeds to small-region unit arithmetic processing in step S230.
[0031] The information obtained as a result of pixel-level computation, including pixels extracted as high-temperature regions, pixels extracted as intrusion regions, and pixels for which frame differences have been calculated, can be used in subsequent stages to calculate feature quantities through computation at the small region level and to calculate feature quantities through computation at the label region level.
[0032] The small-region arithmetic processing in step S230 is comprised of step S231 and is executed by the small-region arithmetic processing unit 22. In step S231, the small-region arithmetic processing unit 22 calculates various feature quantities in small regions based on the pixel-level arithmetic results obtained in the preceding steps S221 to S223 and the time-series data of the input image stored in the image memory 10.
[0033] Specifically, the small-region processing unit 22 calculates three feature quantities, such as feature quantities 1 to 3, on a small-region basis. Here, a small region corresponds to an area grouped by a few adjacent pixels, such as a 3x3 pixel area or a 5x5 pixel area.
[0034] <Feature 1> The small-region processing unit 22 calculates the ratio of pixels indicating high temperature within a small region as feature quantity 1, based on the high-temperature region extraction process at the pixel level in step S221. For example, if the small region is 3x3, a small region containing 5 pixels indicating high temperature is: Feature 1 = 5 / 9 It is calculated as follows.
[0035] <Feature 2> The small-region processing unit 22 calculates the ratio of the number of pixels determined to be invading objects within the small region as feature quantity 2, based on the pixel-level invading region extraction process in step S222. For example, if the small region is 3x3, a small region with 7 pixels determined to be invading objects is: Feature 2 = 7 / 9 It is calculated as follows.
[0036] <Feature 3> The small-region processing unit 22 calculates the ratio of pixels within a small region that show a difference value greater than or equal to a certain value as feature quantity 3, based on the pixel-level frame difference processing performed in step S223. For example, if the small region is 3x3 and the certain difference value is 10, then a small region in which there are 3 pixels showing a difference value greater than or equal to the certain value is: Feature 3 = 3 / 9 It is calculated as follows. Then, the process proceeds to step S240, which involves calculations on a label area basis.
[0037] The information regarding various features within a small region, obtained as a result of computational processing at the small region level, can be used in subsequent processes such as labeling and statistical calculation of features at the label region level.
[0038] The above-mentioned features 1 to 3 are just examples, and the small-area arithmetic processing unit 22 can calculate multiple highly versatile features that indicate brightness changes, correlations, etc., based on the time-series data of the input image stored in the image memory 10. In other words, the small-area arithmetic processing unit 22 can be configured to calculate highly versatile features related to fire, depending on the purpose of fire detection or the monitoring environment.
[0039] Furthermore, the small-region processing unit 22 can calculate spatial features or time-varying features in a small region based on the time-series data of the input image stored in the image memory 10, and store the calculation results for each feature over a predetermined period from the past to the present as time-series data in the time-series storage area.
[0040] Each time-series data of spatial features or time-varying features in a small region, stored as multiple features in the time-series memory area, will be used in the subsequent steps S241 and S242.
[0041] The calculation process performed in step S240 on a label region basis consists of steps S241 to S243 and is executed by the label region calculation processing unit 23. In step S241, the label region calculation processing unit 23 performs a labeling process that integrates adjacent small regions that satisfy predetermined conditions, based on the calculation results of various feature quantities obtained in the preceding step S231.
[0042] For example, the label region processing unit 23 extracts subregions that satisfy the following two conditions based on multiple feature quantities stored in the time-series memory region. <Condition 1> The image memory 10 stores the past five input images, and the time-series data for which the above-mentioned feature quantity 1 was calculated is referenced. For example, small regions in which feature quantity 1 was detected at least once out of the five images are extracted. <Condition 2> The image memory 10 stores the past five input images, and the time-series data for which the feature quantity 2 described above was calculated is referenced. For example, small regions in which feature quantity 2 was detected at least once out of the five are extracted.
[0043] The label area calculation processing unit 23 then sets the sub-regions that satisfy both condition 1 and condition 2 to 1. Furthermore, the label area calculation processing unit 23 performs a labeling process to merge adjacent sub-regions that are set to 1.
[0044] Figure 3 is an explanatory diagram of the labeling process by the label area calculation processing unit 23 according to Embodiment 1 of this disclosure. In Figure 3(A), the portions extracted from adjacent small areas that satisfy predetermined conditions based on the processing results of the small area calculation processing unit 22 are shown as white small areas.
[0045] Furthermore, Figure 3(B) shows the state in which the four islands labeled "1" to "4" have been extracted as fire detection processing areas by applying a labeling process that integrates the white-out small areas.
[0046] Next, in step S242, the label region calculation processing unit 23 extracts candidate regions and feature statistics necessary for fire determination on a label basis by calculating statistical quantities for each time series data of spatial features or time-varying features in the sub-regions stored in the time series memory region for each of the extracted fire determination processing regions.
[0047] Specifically, the label region calculation processing unit 23 calculates, for example, the following statistics 1 and statistics 2 on a label region basis, based on the feature quantity 1 related to the high-temperature region described above.
[0048] <Statistics 1> Based on time-series data related to feature 1 in subregions stored in the time-series memory area, the area of high-temperature regions within each label region is calculated by counting the number of subregions that were identified as high-temperature regions at a ratio of, for example, two or more times in the past five trials.
[0049] <Statistics 2> Based on the time-series data related to feature 1 in the subregions stored in the time-series memory area, the number of subregions that were commonly identified as high-temperature regions in the past five trials is counted for each label region. The area ratio of high-temperature regions within the label region is calculated by normalizing this count by the number of subregions included in the label region.
[0050] Furthermore, the label region processing unit 23 calculates, for example, the following statistical quantity 3 on a label region basis, based on the feature quantity 3 related to the frame difference described above.
[0051] <Statistics 3> For each label region, the number of subregions that satisfy the condition of feature 3 is counted, and by normalizing this by the number of subregions included in the label region, the area ratio of regions exhibiting feature 3 within the label region is calculated.
[0052] The statistics 1 to 3 mentioned above are just examples. The label region calculation unit 23 can extract candidate regions for identifying flame regions and calculate statistical values of features suitable for fire determination, based on various features related to spatial features or time-varying features calculated by the small region calculation unit 22. In other words, the label region calculation unit 23 can be configured to extract candidate regions and calculate statistical values of features suitable for fire determination, depending on the purpose of fire detection or the monitoring environment.
[0053] More specifically, the label region calculation processing unit 23 can perform labeling based on the spatial features or time-varying features in the small regions calculated by the small region calculation processing unit 22, and calculate various feature statistics for each label region.
[0054] Next, in step S243, the label area calculation processing unit 23 identifies the flame area based on the statistical quantities for each label area calculated in step S242. The specific process for identifying the flame area in step S243 will be described in detail in Embodiments 2 and 3.
[0055] The features of the fire detection device according to this embodiment 1 described above can be summarized as follows: <Feature 1> The fire detection device according to this embodiment 1 is equipped with the following three processing units. • Pixel calculation processing unit that performs calculations on a pixel-by-pixel basis on the input image. • Small-area arithmetic processing unit that performs calculations on a small-area unit that combines multiple pixels. • Label area calculation processing unit that performs calculations on a label area basis obtained by performing labeling processing on a small area basis.
[0056] Specifically, the pixel calculation processing unit performs pixel-level calculations as the first stage of processing on the input image, and the small-region calculation processing unit uses the processing results from the first stage in the pixel calculation processing unit and the input image to perform the small-region-level calculations as the second stage of processing.
[0057] Furthermore, the label region calculation unit extracts the label region as a fire determination processing region by performing a labeling process that integrates adjacent sub-regions that satisfy predetermined conditions based on the processing results of the second stage in the sub-region calculation unit. In addition, the label region calculation unit extracts statistical values of feature quantities for identifying flame regions by performing calculations on a per-label region basis as a third stage of processing for each fire determination processing region.
[0058] This configuration allows for the calculation of multiple highly versatile features for each sub-region, followed by labeling based on the calculation results to identify label regions corresponding to the fire detection processing area. Furthermore, statistical values for spatial or temporally changing features are calculated for each label region, enabling a step-by-step narrowing down of candidate regions. As a result, it becomes possible to achieve highly accurate and robust fire detection processing that is resistant to partial shielding.
[0059] <Feature 2> In the fire detection device according to this embodiment 1, the small area calculation processing unit and the label area calculation processing unit, in particular, have the following functions. The small-region processing unit calculates multiple feature quantities for each small region, such as spatial feature quantities or time-varying feature quantities within the small region, and stores the calculation results for each feature quantity over a predetermined period from the past to the present as time-series data in the time-series storage area.
[0060] The label area calculation processing unit calculates statistical values for time-series data relating to spatial features or time-varying features in each of the fire determination processing areas extracted by performing labeling processing based on time-series data relating to multiple features stored in the time-series memory area.
[0061] In this type of processing, fire detection is performed in the label area without using information about the flame's specific fluctuation frequencies. Therefore, the sampling interval for acquiring images does not generally need to consider the frequency band where flame fluctuation energy is concentrated (for example, the band up to 10 Hz). In other words, the input image can be acquired with a sampling interval that does not retain the entire frequency range below 10 Hz, which is the frequency band where flame fluctuation energy is concentrated.
[0062] More specifically, the sampling interval for acquiring the input image can be set to, for example, 0.5 to 2 seconds. As a result, even with video footage acquired under sampling conditions that generate aliasing noise, high detection accuracy can be achieved.
[0063] Furthermore, by incorporating Feature 2, high accuracy can be achieved even in images where aliasing noise occurs. This allows for an extremely high tolerance for jitter in the image sampling interval, enabling the realization of a fire detection device that can exhibit stable detection performance even when images are acquired via a network with fluctuating traffic.
[0064] As described above, according to Embodiment 1, a fire detection device with robustness capable of handling various monitoring environments can be obtained.
[0065] Embodiment 2. In this second embodiment, a specific configuration for identifying flame regions based on multiple statistical quantities calculated for each label region will be described in detail.
[0066] Figure 4 is an explanatory diagram showing the flow of a series of processes in a fire detection device according to Embodiment 2 of this disclosure. In the explanation of Figure 4 in Embodiment 2, and in the explanation of Figures 5 to 7 in Embodiment 3, which will be described later, the part that performs calculation processing on a small area basis after the pixel-based calculation processing described in Embodiment 1 is combined into a single small area calculation processing step.
[0067] In Figure 4, the series of processes is described in six steps: S410, S420, S430, S440, S450, and S460. Of these, steps S410, S420, and S430 are equivalent to those described in Embodiment 1 above, and are processed with the same configuration as the functional block diagram shown in Figure 1 above.
[0068] On the other hand, steps S440, S450, and S460, which are characteristic parts of this second embodiment, are processed by a new configuration not shown in Figure 1, consisting of N subclassifiers 1 to N, 3 discriminants 1 to 3, and a voting machine 51, respectively. Therefore, steps S410, S420, and S430 will be briefly explained, and steps S440, S450, and S460 will be explained in detail.
[0069] In step S410, the input image captured by the surveillance camera 1 is stored in the image memory 10 at a predetermined sampling period, thereby performing image input processing. Figure 4 illustrates the case where an infrared camera is used as the surveillance camera 1 and an infrared image is obtained as the input image.
[0070] Next, in step S420, calculations are performed on a pixel-by-pixel basis and on a small-region basis to calculate multiple highly versatile feature quantities 1 to L. Then, in step S430, calculations are performed on a label region basis to calculate multiple statistics 1 to M as spatial feature quantities or time-varying feature quantities for each labeled label region.
[0071] Next, in step S440, a process is performed to identify whether each label area is a fire area or a non-fire area using each of the multiple sub-classifiers 30(1) to 30(N). Here, the sub-classifier 30 is configured to identify whether each label area is a fire area or a non-fire area after dividing the conditions necessary for fire determination based on one or more statistics from among the multiple statistics 1 to M calculated by the label area calculation processing unit 23.
[0072] In other words, each of the sub-classifiers 30, while not having high individual identification accuracy, performs identification processing based on appropriately selected statistics so that it can identify specific conditions. It sets a label area identified as a fire area to 1 and a label area identified as a non-fire area to -1. For example, the sub-classifier 30(1) using the previously described statistic 1 and the sub-classifier 30(2) using the statistic 3 can be configured as follows.
[0073] <Small classifier 30(1)> Based on the time transition of statistic 1, which indicates the area of the high-temperature region within the labeled region, if the area fluctuation in the labeled region where the high-temperature region continuously occurs is within a predetermined range, it is identified as a fire region; otherwise, it is identified as a non-fire region.
[0074] <Small Identifier 30(2)> Based on the time evolution of statistic 3, which represents the area ratio of the regions exhibiting feature 3 within the labeled region, if the maximum value is within a predetermined range, it is identified as a fire region; otherwise, it is identified as a non-fire region.
[0075] Furthermore, each sub-classifier 30 is not limited to using only one statistical quantity; it can also use a combination of two or more statistical quantities. In addition, the statistical quantities used and the predetermined ranges set for each sub-classifier 30 can be appropriately set according to the purpose of fire detection or the monitoring environment. Moreover, each sub-classifier 30 can be individually configured to identify not only conditions suitable for extracting fire areas, but also conditions suitable for extracting non-fire areas.
[0076] Many of the statistical quantities used in the subclassifier 30 have some correlation with the area of the label region. Therefore, even within the same subclassifier 30, the value set as a predetermined range may be changed according to the area of the label region, thereby enabling classification processing that takes correlation into account.
[0077] Next, in step S450, each of the multiple discriminants 40(1) to 40(3) is used to determine whether each label area is a fire area or a non-fire area, according to the purpose of fire detection.
[0078] Here, the discriminator 40 is configured to select a group of sub-classifiers consisting of one or more desired sub-classifiers from among a plurality of sub-classifiers 30(1) to 30(N) according to the purpose of fire detection, and to determine whether each label area is a fire area or a non-fire area by performing a logical operation on the identification results of each sub-classifier 30 included in the selected sub-classifier group.
[0079] In other words, each of the classifiers 40 selects a group of subclassifiers according to the purpose of fire detection and performs rigorous discrimination through logical operations to distinguish between fire areas and non-fire areas, thereby suppressing factors that could cause false alarms.
[0080] Next, in step S460, a process is performed to identify the flame area by voting using a single voting machine 51 and a combination of the discrimination results from multiple classifiers 40(1) to 40(3).
[0081] The voting device 51 identifies the flame area by combining the results of a discriminator 40(1) specialized for label areas with a relatively large area, and the results of discriminators 40(2) and 40(3) specialized for label areas with a relatively small area. By using multiple small discriminators 30, which have low accuracy individually, and combining them with multiple discriminators 40, high-precision fire detection performance can be achieved. In this embodiment, three discriminators are used, but the number of discriminators is not limited to this embodiment; it is best to prepare a number that allows for appropriate fire detection.
[0082] The features of the fire detection device according to this second embodiment described above can be summarized as follows: <Feature 1> The fire detection device according to this second embodiment includes the following three components after the label area calculation processing unit described in the first embodiment. Multiple sub-classifiers identify whether each label area is a fire area or a non-fire area based on one or more statistics calculated by the label area calculation processing unit.
[0083] - Multiple classifiers that, depending on the purpose of fire detection, select a group of subclassifiers consisting of one or more desired subclassifiers from among several subclassifiers, and determine whether each label area is a fire area or a non-fire area by performing logical operations on the identification results of each subclassifier included in the selected subclassifier group. • A voting device that identifies the flame area by combining the results of classifications from multiple classifiers.
[0084] Specifically, each of the multiple sub-classifiers is configured by dividing the conditions necessary for fire detection, and is configured to specialize in specific conditions. It may also include a sub-classifier for false alarm detection, in which the conditions necessary for fire detection are set with the aim of not detecting specific false alarm factors as fires.
[0085] Similarly, the system can be configured to include a classifier that detects a specific non-fire.
[0086] In other words, the fire detection device according to this second embodiment has a configuration in which a large number of highly versatile feature quantities are prepared in advance, a large number of effective but not highly discriminative subclassifiers are prepared using statistical quantities calculated from these feature quantities, and voting is performed by multiple classifiers that combine these subclassifiers.
[0087] Therefore, by adopting such a configuration, even a classifier with low accuracy on its own can achieve high-accuracy fire detection performance. In other words, it can improve upon conventional methods that required finding effective features to distinguish between fire and non-fire areas for a specific phenomenon and devising an appropriate algorithm each time.
[0088] <Feature 2> The fire detection device according to this second embodiment can change the combination of sub-classifiers included in the sub-classifier group according to the area of the label area and perform selection, and is configured to perform identification processing between fire areas and non-fire areas using different thresholds according to the area of the label area even within the same sub-classifier.
[0089] Furthermore, by focusing on a certain feature or statistic and using a combination of different classifiers to detect a fire when its value falls within a certain range, it is possible to perform fire detection specifically for a small area, for example.
[0090] As described above, according to Embodiment 2, a fire detection device with robustness capable of handling various monitoring environments can be obtained.
[0091] Embodiment 3. In the previous embodiment 2, we described a case where a designer designs multiple sub-classifiers 30 and multiple discriminants 40, and appropriate discrimination processing is performed to identify fire areas and non-fire areas. In contrast, embodiment 3 will describe in detail a case in which machine learning-based discrimination processing is used.
[0092] The following three methods are possible for applying machine learning-based discrimination processing. (1) A first machine learning classifier that uses the identification results of multiple subclassifiers 30 as input parameters to determine whether each labeled area is a fire area or a non-fire area. (2) A second machine learning classifier that uses multiple statistical quantities calculated by the label area calculation processing unit 23 as input parameters to determine whether each label area is a fire area or a non-fire area. (3) A classifier that combines a plurality of classifiers 40 designed by the designer as described in Embodiment 2 above, a first machine learning classifier, and a second machine learning classifier.
[0093] Therefore, each of (1) to (3) will be explained individually below using diagrams. <(1) Regarding the first machine learning discriminant> Figure 5 is an explanatory diagram showing the flow of a series of processes in a fire detection device using the first machine learning discriminant 61 according to Embodiment 3 of this disclosure. The four steps S410, S420, S430, and S440 in Figure 5 are the same as the four steps S410, S420, S430, and S440 in Figure 4, and their explanation is omitted.
[0094] On the other hand, the distinctive features of Figure 5 in this third embodiment are steps S550 and S560, which are processed by a new configuration not shown in Figure 1, consisting of the first machine learning discriminator 61 and the voting machine 52, respectively. Therefore, steps 550 and S560 will be described in detail below.
[0095] In step S550, the first machine learning classifier 61 is used to determine whether each label area is a fire area or a non-fire area, using the identification results from multiple subclassifiers 30(1) to 30(N) as input parameters.
[0096] Here, the first machine learning classifier 61 performs fire detection using a convolutional neural network (CNN). The first machine learning classifier 61 has been trained based on training data previously obtained as identification results from multiple subclassifiers 30(1) to 30(N).
[0097] The CNN used is, for example, a CNN with the number of input layers = number of subclassifiers N, the number of hidden layers = number of input layers, and the number of output layers = 1. The input layer receives the classification results from multiple subclassifiers 30(1) to 30(N), and the output layer outputs the classification result.
[0098] Furthermore, the first machine learning discriminator 61 can also be composed of multiple CNNs trained on different training data and initial parameters. For example, it is possible to use a separate CNN for each specific monitoring environment, or to use CNNs with different discriminant characteristics by changing the initial values. Alternatively, separate CNNs could be used for specific false alarm factors, separate from the training for identifying fire areas.
[0099] Note that neural networks themselves are a well-known technology, so a detailed explanation will be omitted.
[0100] Next, in step S560, a single voting machine 52 is used to identify the flame region by voting on a combination of the discrimination results from the first machine learning classifier 61. The voting machine 52 identifies the flame region by voting on a combination of the discrimination results from the first machine learning classifier 61, which is composed of multiple CNNs. By combining multiple small classifiers 30, which have low accuracy individually, with the first machine learning classifier 61, the voting machine 52 can achieve high-accuracy fire discrimination performance.
[0101] <(2) Regarding the second machine learning discriminant> Figure 6 is an explanatory diagram showing the flow of a series of processes in a fire detection device using the second machine learning discriminant 62 according to Embodiment 3 of this disclosure. The three steps S410, S420, and S430 in Figure 6 are the same as the three steps S410, S420, and S430 in Figure 4, and their explanation is omitted.
[0102] On the other hand, the distinctive parts of Figure 6 in this third embodiment are steps S650 and S660, which are processed by a new configuration not shown in Figure 1, consisting of a second machine learning discriminator 62 and a voter 53, respectively. Therefore, steps 650 and S660 will be described in detail below.
[0103] In step S650, the second machine learning discriminator 62 is used to determine whether each label region is a fire region or a non-fire region, using multiple statistical quantities calculated by the label region calculation processing unit 23 as input parameters.
[0104] Here, the second machine learning discriminator 62 performs fire detection using a CNN, similar to the first machine learning discriminator 61. The second machine learning discriminator 62 has been trained based on pre-acquired training data, which is calculated by the label region processing unit 23 as multiple statistical quantities.
[0105] The CNN used is, for example, a CNN with input layers = number of statistics M, hidden layers = number of input layers, and output layers = 1. Multiple statistics calculated by the label region processing unit 23 are input to the input layer, and the classification result is output from the output layer.
[0106] Furthermore, the second machine learning discriminator 62 can also be composed of multiple CNNs that have been individually trained based on different training data and initial parameters. For example, it is possible to use a separate CNN for each specific monitoring environment, or to use CNNs with different discriminant characteristics by changing the initial values. Alternatively, it is possible to use a separate CNN for specific false alarm factors, separate from the training for identifying fire areas.
[0107] Note that neural networks themselves are a well-known technology, so a detailed explanation will be omitted. Furthermore, the differences between the first machine learning discriminant 61 shown in Figure 5 and the second machine learning discriminant 62 shown in Figure 6 will be explained in detail in section (3) below.
[0108] Next, in step S660, a single voting device 53 is used to identify the flame region by voting on a combination of the discrimination results from the second machine learning classifier 62. By identifying the flame region through voting on a combination of the discrimination results from the second machine learning classifier 62, which is composed of multiple CNNs, the voting device 53 can achieve high accuracy in fire discrimination performance using multiple statistical measures.
[0109] <(3) Regarding the combination of classifier 40 and machine learning classifiers 61 and 62> Figure 7 is an explanatory diagram showing the flow of a series of processes in a fire detection device that uses a discriminator 40 according to Embodiment 3 of this disclosure in combination with a first machine learning discriminator 61 and a second machine learning discriminator 62.
[0110] The six steps S410, S420, S430, S440, S450, and S460 in Figure 7 are identical to the six steps S410, S420, S430, S440, S450, and S460 in Figure 4. Also, the two steps S550 and S560 in Figure 7 are identical to the two steps S550 and S560 in Figure 5. Furthermore, the two steps S650 and S660 in Figure 7 are identical to the two steps S650 and S660 in Figure 6.
[0111] On the other hand, a distinctive feature of Figure 7 in this embodiment 3 is step S770, which is processed by the fire detector 71 using a new configuration not shown in Figure 1. Therefore, step 770 will be explained in detail.
[0112] The series of six steps, S410, S420, S430, S440, S450, and S460, is a fire detection method using a discriminator 40 designed by the designer, as described in the previous embodiment 2. Therefore, the fire detection result will differ depending on the design of the multiple sub-discriminators 30 and the multiple discriminators 40 by the designer.
[0113] Furthermore, the series of six steps S410, S420, S430, S440, S550, and S560 constitutes a fire detection method using the first machine learning discriminator 61 described in (1) above. Therefore, the fire detection result will differ depending on the learned content.
[0114] Furthermore, the series of five steps S410, S420, S430, S650, and S660 constitutes a fire detection method using the second machine learning discriminator 62 described in (2) above. Therefore, the fire detection result will differ depending on the learned content.
[0115] Furthermore, comparing the discriminator 40, the first machine learning discriminator 61, and the second machine learning discriminator 62, they each have the following characteristics.
[0116] The performance of the classifier 40 depends on the designer, but it has the characteristic of being able to improve fire detection performance even with limited data, depending on the designer's experience and the characteristics of the installation environment. It is also possible to intentionally design the classifier 40 to have strict fire detection characteristics in order to suppress false alarms.
[0117] Furthermore, the first machine learning classifier 61 can obtain fire discrimination performance that takes various situations into account based on training data created from measured data. In addition, since the discrimination results of multiple sub-classifiers 30 are used as input parameters, the performance will be in accordance with the design of the sub-classifiers, and it has the characteristic of being able to perform fire discrimination based on actual measurements while also being based on the designer's experience. Qualitatively, a classifier with such a configuration has characteristics intermediate between the second machine learning classifier 62 and the classifier 40.
[0118] Furthermore, the second machine learning classifier 62 obtains fire discrimination performance that takes various situations into account based on training data created from measured data. In addition, because it uses multiple statistical quantities calculated by the label area calculation processing unit 23 as input parameters, it has fire discrimination characteristics based on logic outside the designer's experience. Qualitatively, a classifier with this configuration has reduced fire discrimination performance, but it has the fastest fire discrimination performance.
[0119] In other words, none of the discriminator 40, the first machine learning discriminator 61, or the second machine learning discriminator 62 can obtain a perfect result that accurately distinguishes between fire and non-fire, but each has the different characteristics described above. Therefore, the configuration shown in Figure 7 includes a fire discriminator 71 that performs fire discrimination processing in the final stage, taking into account the voting results based on the discrimination processing of the discriminator 40, the first machine learning discriminator 61, and the second machine learning discriminator 62.
[0120] In step S770, the final step involves using the fire discriminator 71 to determine whether each label area is a fire area or a non-fire area, taking into account the voting results based on the respective discriminative processes of the discriminator 40, the first machine learning discriminator 61, and the second machine learning discriminator 62.
[0121] The characteristics of the fire detection device that performs the series of processes shown in Figure 7 can be summarized as follows: <Feature 1> The fire detection device 71 will receive the following three detection results via three different detection devices and voting devices. The voting results from the voting machine 52, based on the respective discrimination results from the first machine learning discriminator 61, are provided to the fire discriminator 71 as the first discrimination result. The voting results from the voting machine 53, based on the respective discrimination results from the second machine learning discriminator 62, are provided to the fire discriminator 71 as the second discrimination result. The voting results from the voting device 51, based on the individual discrimination results from the multiple discrimination devices 40, are provided to the fire discrimination device 71 as a third discrimination result.
[0122] The fire detector 71 can then identify the flame area based on a majority vote using the first, second, and third determination results.
[0123] Here, the preliminary steps in which individual discrimination results are obtained from three types of voting by voting machines 51, 52, and 53 correspond to the first process section, and the subsequent steps in which the fire discrimination device 71 generates a final judgment result from the three discrimination results correspond to the second process section.
[0124] In other words, the fire detection device equipped with the processing configuration shown in Figure 7 according to this embodiment 3 prepares a large number of highly versatile features in advance, prepares a large number of effective but not highly accurate subclassifiers using statistics calculated from these features, and generates a first classification result by performing a voting process on the classification result of the first machine learning classifier 61, which uses the output of these subclassifiers as input parameters.
[0125] Furthermore, by preparing a large number of highly versatile features in advance and performing a voting process on the classification results of a second machine learning classifier 62 that uses statistical quantities calculated from these features as input parameters, a second classification result can be generated.
[0126] Furthermore, by preparing a large number of highly versatile features in advance, and using the statistics calculated from these features to prepare a large number of effective but less accurate subclassifiers, and then performing a voting process on the classification results of multiple classifiers 40 that combine these subclassifiers, a third classification result can be generated.
[0127] Based on the three types of discrimination results (first, second, and third), the flame area can finally be identified. As a result, even if each individual discriminator cannot reach the target accuracy, the combination of discrimination results from discriminators with different characteristics can improve the accuracy of fire detection performance.
[0128] As described above, according to Embodiment 3, a fire detection device with robustness capable of handling various monitoring environments can be obtained.
[0129] Embodiment 4. In this fourth embodiment, we will describe a method for setting the temperature threshold Tth used to extract high-temperature regions from the input image when an infrared thermography camera is used as the surveillance camera 1.
[0130] For example, when using fire detection devices in outdoor applications such as tunnels or outdoor plants, contamination of the optical system during monitoring can significantly affect detection performance. Therefore, optical fire detectors and similar devices have a function to assess the degree of contamination of the sensor's window material and compensate for the detection sensitivity.
[0131] When applying the fire detection device according to this disclosure, which uses infrared thermography as a surveillance camera 1 to acquire information on the temperature of the monitored environment and performs fire monitoring by image processing, if the lens or window material becomes contaminated, a lower temperature than the actual temperature may be observed.
[0132] Furthermore, the contamination of the lens and window material is not uniform, and the degree of contamination may differ between the central and peripheral areas. In other words, when using infrared thermography as a surveillance camera 1, an imaging optical system is employed, so the degree of contamination will differ depending on the position of the image. However, accurately determining the distribution of this contamination is extremely difficult.
[0133] To address these challenges, Embodiment 4 has a technical feature in that, assuming in advance that the contamination condition will deteriorate, the temperature threshold Tth used to extract pixels constituting the high-temperature region from the input image as the pixels constituting the target area for calculation processing is set to a value robust to contamination. The specific setting method is described below.
[0134] The process of extracting pixels that constitute the high-temperature region from the input image is performed by the pixel calculation processing unit 21.
[0135] First, an appropriate temperature threshold that allows for stable extraction of the fire area while the lens is clean is experimentally obtained, for example, during the design process. This appropriate temperature threshold will be referred to as the "set temperature" below. In other words, the "set temperature" is defined as the state in which the infrared thermography optical system is uncontaminated, where the transmittance of the wavelength band of the infrared thermography optical system is 100%, and the appropriate temperature threshold that allows for extraction of the fire area when imaging with infrared thermography at this 100% state.
[0136] Next, the infrared thermography camera is set to a temperature environment equivalent to the maximum ambient temperature at which it can be installed in the monitoring environment. This maximum temperature will be referred to as the "ambient temperature" below.
[0137] Typically, the contamination of optical equipment installed in outdoor environments is managed and set based on the transmittance of visible light wavelengths for management purposes.
[0138] On the other hand, contamination of these optical instruments is caused by the accumulation of fine particles such as soot, but for the same particle size, the spectral transmittance increases as the wavelength increases, due to the increase in diffraction.
[0139] In other words, by controlling the transmittance in the wavelength range of infrared thermography (which varies depending on the material, but is generally 3μm to 5μm, or 8μm to 14μm), it is possible to ensure monitoring performance down to lower transmittances in the visible light wavelength range.
[0140] Based on this management method, in this embodiment 4, an appropriate temperature threshold Tth is set using the exponential function of equation (1) below to extract high-temperature regions from the input image. Tth = ((Ea - Eb) × Z / 100 + Eb) x (1)
[0141] Here, each symbol in equation (1) above has the following meaning: Ea: Total optical energy at the set temperature Eb: Total optical energy at ambient temperature Z: Transmittance when contaminated to the acceptable limit (i.e., transmittance at the acceptable limit of contaminated state for the optical system) Z% (where Z is a positive number less than or equal to 100) X: Constant for calculating the temperature threshold from the total optical energy. Furthermore, the temperature threshold Tth can also be determined by means other than equation (1) above, based on the total optical energy Ea at the set temperature and the total optical energy Eb at the ambient temperature.
[0142] For example, if Z=50% and the temperature threshold Tth is set according to equation (1) above, continuous monitoring is possible up to 50% contamination, even if the contamination status changes, without the need for sensitivity compensation.
[0143] Therefore, by setting an appropriate temperature threshold Tth using equation (1) above, the impact of optical system contamination during monitoring on detection performance can be suppressed, and the ability to extract high-temperature regions until a predetermined contamination state is reached can be maintained without sensitivity compensation.
[0144] As described above, according to Embodiment 4, a fire detection device with robustness capable of handling various monitoring environments can be obtained. [Explanation of symbols]
[0145] 1 Surveillance camera, 10 Image memory, 20 Processing unit, 21 Pixel processing unit, 22 Small area processing unit, 23 Label area processing unit, 30 Small classifier, 40 Discriminant, 51, 52, 53 Voting machine, 61 First machine learning discriminant, 62 Second machine learning discriminant, 71 Fire discriminant.
Claims
1. A fire detection device that identifies the flame area by performing calculations on input images captured by a surveillance camera, A small-region arithmetic processing unit that performs arithmetic processing on a small-region unit, which is a group of pixels that make up the input image, A label region calculation processing unit performs calculations on a label region basis for each of the label regions obtained by performing label processing to integrate adjacent small regions that satisfy predetermined conditions based on the processing results of the small region calculation processing unit by the small region calculation processing unit, Equipped with, The aforementioned small-area arithmetic processing unit is The system has a time-series storage area that stores time-series data for each of the multiple feature quantities calculated by the small-region unit calculation process in order to identify the flame region, For each of the aforementioned multiple features, spatial features or time-varying features in a small region are calculated, and the calculation results for each feature over a predetermined period from the past to the present are stored as time-series data in the time-series storage area. The label area calculation processing unit is: The label region obtained by performing the labeling process based on the time-series data relating to the plurality of feature quantities stored in the time-series memory region is extracted as the fire determination processing region. Calculate the statistical quantities of the time-series data relating to the spatial features or time-varying features in each of the extracted fire determination processing areas. Flame region identification unit identifies the flame region by performing machine learning discrimination processing based on a plurality of statistical quantities calculated by the label region calculation processing unit. A fire detection device that also includes additional features.
2. The flame region identification unit is, The conditions necessary for fire determination are divided based on one or more statistical quantities from among the multiple statistical quantities calculated by the label area calculation processing unit, and a plurality of sub-identifiers are configured to identify whether each label area is a fire area or a non-fire area, A first machine learning classifier is trained based on pre-acquired training data obtained as identification results from the aforementioned multiple subclassifiers, and uses the identification results as input parameters to determine whether each labeled area is a fire area or a non-fire area. A fire detection device according to claim 1, comprising:
3. The flame region identification unit is, A second machine learning classifier is trained based on pre-acquired training data, with multiple statistical quantities calculated by the label region calculation processing unit, and uses these statistical quantities as input parameters to determine whether each label region is a fire region or a non-fire region. A fire detection device according to claim 1, comprising:
4. The flame region identification unit is, The conditions necessary for fire determination are divided based on one or more statistical quantities from among the multiple statistical quantities calculated by the label area calculation processing unit, and a plurality of sub-identifiers are configured to identify whether each label area is a fire area or a non-fire area, Multiple classifiers that, according to the purpose of fire detection, select a group of subclassifiers consisting of one or more desired subclassifiers from the group of subclassifiers, and determine whether each of the label areas is a fire area or a non-fire area by performing logical operations on the identification results of each subclassifier included in the selected group of subclassifiers, A first machine learning classifier, which is trained based on pre-acquired training data obtained as identification results by the aforementioned multiple subclassifiers, and which uses the identification results as input parameters to determine whether each labeled area is a fire area or a non-fire area, A second machine learning classifier, which is trained based on pre-acquired training data as a plurality of statistics calculated by the label area calculation processing unit, and which uses the statistics as input parameters to determine whether each label area is a fire area or a non-fire area, A voting device that identifies the flame region by voting on a combination of the discrimination results from the multiple discriminators, the discrimination result from the first machine learning discriminator, and the discrimination result from the second machine learning discriminator. A fire detection device according to claim 1, comprising:
5. The first machine learning discriminant is composed of a plurality of first machine learning discriminants that have undergone different training. The second machine learning discriminant is composed of multiple second machine learning discriminants that have undergone different training. The voting device has a first process section and a second process section to identify the flame region, The first process unit is, A first voting device that identifies the flame region as a first discrimination result based on the discrimination results of each of the plurality of first machine learning discriminators, A second voting machine that identifies the flame region as a second discrimination result based on the discrimination results of each of the plurality of second machine learning discriminators, A third voting device identifies the flame region as a third determination result based on the determination results of each of the aforementioned multiple determination devices, and It consists of, The second process unit is comprised of a fire detector that identifies the flame region based on a majority vote using the first determination result, the second determination result, and the third determination result. The fire detection device according to claim 4.
Citation Information
Patent Citations
Object detection device / method and object monitoring system
JP1997293185A
Smoke detection device
JP2011215806A
Authentication device and authentication method
JP2015204030A
Smoke detection device and smoke detection method
JP2016110262A
Smoke detection device
JP2021087084A
