Flame detection device, flame detection method, image generation device, movement confirmation system, image generation method, and program

Abstract

This flame detection device comprises: an acquisition unit that acquires a plurality of target images in which an object under observation has been imaged in a time series; an extraction unit that ext

Description

TITLE OF INVENTION: FLAME DETECTION DEVICE, FLAME DETECTION METHOD, IMAGE GENERATION DEVICE, MOVEMENT CONFIRMATION SYSTEM, IMAGE GENERATION METHOD, AND PROGRAM TECHNIC AL FIELD

[0001] The present invention relates to a flame detection device, a flame detection method, an image generation device, a movement confirmation system, an image generation method, and a program. Priority is claimed on Japanese Patent Application No. 2022-046926, filed March 23, 2022, Japanese Patent Application No. 2022-050049, filed March 25, 2022, and Japanese Patent Application No. 2022-052390, filed March 28, 2022, the contents of which are incorporated herein by reference. BACKGROUND ART

[0002] Techniques of detecting outbreak of a fire using an image are known (for example, Patent Literature 1 to 3). A moving object in images is extracted from a plurality of images captured in a time series, and it is determined that the extracted moving object is a flame when the moving object moves like a flame. For example, when coordinates of the position of the center of gravity of the moving object change periodically or increase monotonously or decrease monotonously, it is determined that the moving object is not a flame but a red light of which a light emitting state changes periodically or a moving object such as a vehicle moving in one direction. By excluding a moving object moving unlike a flame from flames, it is possible to prevent a moving object such as a vehicle of which a red light is turned on from being erroneously detected as a flame and to improve detection accuracy. However, a flame may move. For example, when a fire breaks out in a moving vehicle, flames can move along with the vehicle. When such a moving fire is imaged, it is preferable to correctly detect that a flame is shown in an image.

[0004] Whether an object like a flame is shown in an image is determined for each pixel, and a region which is a group of pixels like a flame is extracted. When the shape of the extracted region is far from a circular shape, it is determined that a flame is shown in the region. For example, circularity (=4kS / La2) can be applied as an index indicating a degree of roundness. Here, S denotes an area of the region, and L denotes a length of an outer circumference of the region. By excluding an object close to a circular shape from a flame, it is possible to prevent a light emitting object such as a revolving light from being erroneously detected as a flame and to improve detection accuracy.

[0005] However, a flame cannot be accurately detected in a determination based on circularity. For example, some light emitting objects may have a shape close to a rectangular shape like a fluorescent light. In an image in which such a rectangular light emitting object emits light is imaged, the shape of the region in which light is emitted may be far from a circular shape and may be erroneously detected as a flame. When a flame is detected, it is preferable to prevent a light emitting object having a shape such as a rectangular shape other than a circular shape from being erroneously detected as a flame.

[0006] A flame detection system that detects a fire using an image is installed in various sites. When a flame detection system is installed, it is preferable to confirm whether the flame detection system in the site is operating correctly in order to accurately detect a fire. However, it is difficult to cause a fire to actually break out in the site and to confirm an operation using an image in which the outbreak of the fire is imaged. Accordingly, as a method of reproducing a fire in an actual space, for example, Patent Literature 4 discloses that a non-fire image and a smoke model of a three-dimensional shape are provided in a virtual space, and a synthetic image in which a smoke image is provided in a direction in which it seems to be natural is generated while changing the direction of the smoke model.

[0007] However, since a three-dimensional shape model is used in Patent Literature 4, it is difficult to generate a synthetic image without using a device with a heavy processing load and high specifications, and there is a problem in that knowledge about a fire is required for a tester to reproduce appropriate fire situations according to places. Citation List Patent Document

[0008] Patent Documentl: Japanese Patent No. 6140599 Patent Document2: Japanese Unexamined Patent Application, First Publication No. H8-305980 Patent Documents: Japanese Unexamined Patent Application, First Publication No. 2020-021300 Patent Document4: Japanese Unexamined Patent Application, First Publication No. 2021-149378 SUMMARY OF INVENTION Technical Problem The present disclosure was invented in consideration of the aforementioned circumstances and an objective thereof is to provide a flame detection device, a flame detection method, an image generation device, a movement confirmation system, an image generation method, and a program in a flame detection system and a flame detection system that can detect a flame with high accuracy. Solution to Problem

[0010] According to a first aspect of the present disclosure, a flame detection device is provided including: an acquisition unit configured to acquire a plurality of target images in which a monitoring target is captured in a time series; an extraction unit configured to extract a flame candidate region from each target image acquired by the acquisition unit; and a determination unit configured to calculate an index indicating that an object shown in the flame candidate region moves like a flame using a degree to which an amount of movement of a tracking point set in the flame candidate region fluctuates with respect to an average value of the amount of movement and to determine whether a flame is shown in the flame candidate region on the basis of the calculated index.

[0011] A second aspect of the present disclosure provides the flame detection device according to the first aspect, wherein the determination unit calculates an optical flow of a pixel included in the flame candidate region as the amount of movement.

[0012] A third aspect of the present disclosure provides the flame detection device according to the first or second aspect, wherein the determination unit calculates the index using at least one of a first number of crossings in which the amount of movement changes from a value less than the average value to a value greater than the average value and a second number of crossings in which the amount of movement changes from a value greater than the average value to a value less than the average value.

[0013] A fourth aspect of the present disclosure provides the flame detection device according to any one of the first to third aspects, wherein the determination unit calculates the amount of movement and the average value in a first direction in the target images and a second direction other than the first direction and determines whether a flame is shown in the flame candidate region on the basis of the index in each of the directions.

[0014] A fifth aspect of the present disclosure provides the flame detection device according to any one of the first to fourth aspects, wherein the extraction unit extracts a first candidate region on the basis of color information included in the plurality of target images, the extraction unit determines whether the first candidate region in a first image and the first candidate region in a second image are based on the same object on the basis of the first candidate region in the first image extracted from the first image out of the plurality of target images and the first candidate region in the second image extracted from the second image which is captured after the first image out of the plurality of target images, and determines the first candidate region in the first image and / or the first candidate region in the second image as a second candidate region when the two first candidate regions are based on the same object, and the extraction unit determines the flame candidate region from the second candidate region.

[0015] A sixth aspect of the present disclosure provides the flame detection device according to the fifth aspect, wherein the extraction unit calculates a degree to which the second candidate region is likely to be a flame on the basis of frequency characteristics in which a color in the second candidate region changes and determines whether the second candidate region is set to be the flame candidate region on the basis of the calculated degree of being like a flame.

[0016] According to a seventh aspect of the present disclosure, a flame detection method is provided that is performed by a flame detection device which is a computer, the flame detection method including: causing an acquisition unit to acquire a plurality of target images in which a monitoring target is captured in a time series; causing an extraction unit to extract a flame candidate region from each target image acquired by the acquisition unit; and causing a determination unit to calculate an index indicating that an object shown in the flame candidate region moves like a flame using a degree to which an amount of movement of a tracking point set in the flame candidate region fluctuates with respect to an average value of the amount of movement and to determine whether a flame is shown in the flame candidate region on the basis of the calculated index.

[0017] According to an eighth aspect of the present disclosure, a program is provided causing a flame detection device which is a computer to perform: acquiring a plurality of target images in which a monitoring target is captured in a time series; extracting a flame candidate region from each target image; and calculating an index indicating that an object shown in the flame candidate region moves like a flame using a degree to which an amount of movement of a tracking point set in the flame candidate region fluctuates with respect to an average value of the amount of movement and determining whether a flame is shown in the flame candidate region on the basis of the calculated index. According to a ninth aspect of the present disclosure, a flame detection device is provided including: an acquisition unit configured to acquire a target image in which a monitoring target is imaged; an extraction unit configured to extract a flame candidate region from the target image acquired by the acquisition unit; and a determination unit configured to calculate an index indicating shape complexity in the flame candidate region using a statistic of angles formed by contour lines connecting pixel groups constituting a contour of the flame candidate region and to determine whether a flame is shown in the flame candidate region on the basis of the calculated index.

[0019] A tenth aspect of the present disclosure provides the flame detection device according to the ninth aspect, wherein the determination unit calculates the index using an angle formed by line segments connecting three pixels selected from a pixel group constituting the contour of the flame candidate region along the contour as an angle formed by the contour lines and using the statistic of angles formed by each of a plurality of the line segments identified for each combination of the selected pixels.

[0020] An eleventh aspect of the present disclosure provides the flame detection device according to the tenth aspect, wherein the determination unit uses an angle formed by line segments connecting three pixels which are not adjacent and which are selected from pixel groups constituting the contour of the flame candidate region along the contour as the angle formed by the contour line.

[0021] A twelfth aspect of the present disclosure provides the flame detection device according to the eleventh aspect, wherein the determination unit calculates an angle formed by connecting three pixels which are separated by N pixels (where N is an integer equal to or greater than 2) out of the pixel groups constituting the contour along the contour as the angle formed by the contour lines.

[0022] A thirteenth aspect of the present disclosure provides the flame detection device according to any one of the ninth to twelfth aspects, wherein the determination unit calculates a ratio of the number of angles which are modes to the total number of angles formed by the contour lines as the index.

[0023] A fourteenth aspect of the present disclosure provides the flame detection device according to any one of the ninth to twelfth aspects, wherein the acquisition unit acquires a plurality of target images in which the monitoring target is captured in a time series, and the determination unit calculates a deviation of angles which are modes out of the angles formed by the contour lines corresponding to each of the plurality of target images as the index.

[0024] A fifteenth aspect of the present disclosure provides the flame detection device according to any one of the ninth to fourteenth aspects, wherein the extraction unit extracts the flame candidate region on the basis of color information in an image.

[0025] According to a sixteenth aspect of the present disclosure, a flame detection method is provided that is performed by a flame detection device which is a computer, the flame detection method including: causing an acquisition unit to acquire a target image in which a monitoring target is imaged; causing an extraction unit to extract a flame candidate region from the target image acquired by the acquisition unit; and causing a determination unit to calculate an index indicating shape complexity in the flame candidate region using a statistic of angles formed by contour lines connecting pixel groups constituting a contour of the flame candidate region and to determine whether a flame is shown in the flame candidate region on the basis of the calculated index.

[0026] According to a seventeenth aspect of the present disclosure, a program is provided causing a flame detection device which is a computer to perform: acquiring a target image in which a monitoring target is imaged; extracting a flame candidate region from the acquired target image; and calculating an index indicating shape complexity in the flame candidate region using a statistic of angles formed by contour lines connecting pixel groups constituting a contour of the flame candidate region and determining whether a flame is shown in the flame candidate region on the basis of the calculated index.

[0027] According to an eighteenth aspect of the present disclosure, an image generation device is provided including: an acquisition unit configured to acquire a target image in which a target region is imaged; and a generation unit configured to generate a synthetic image in which a smoke image is synthesized with the target image acquired by the acquisition unit, wherein the synthetic image is used to determine whether a smoke detection device which is provided in the target region and which determines whether smoke is being generated in the target region on the basis of an image is operating correctly.

[0028] A nineteenth aspect of the present disclosure provides the image generation device according to the eighteenth aspect, wherein the generation unit generates the synthetic image with a sum value of a pixel value in the target image and a value obtained by multiplying a pixel value in the smoke image by a predetermined multiple as a pixel value.

[0029] A twentieth aspect of the present disclosure provides the image generation device according to the nineteenth aspect, wherein the generation unit sets the predetermined multiple to a positive value when the smoke image is represented as white smoke and sets the predetermined multiple to a negative value when the smoke image is represented as black smoke.

[0030] A twenty-first aspect of the present disclosure provides the image generation device according to the eighteenth aspect, further including: a display unit configured to display the target image and a guide for guiding a color of smoke based on an object which is a fire source; and an input unit configured to receive an input of an operation from a user, wherein the generation unit generates the synthetic image in which a smoke image of a color designated by the operation input to the input unit is synthesized with the target image.

[0031] A twenty-second aspect of the present disclosure provides the image generation device according to the eighteenth aspect, wherein depth information indicating a depth of the target region is included in the target image, and the generation unit generates the synthetic image in which a smoke image with a size corresponding to the depth information is synthesized with the target image.

[0032] According to a twenty-third aspect of the present disclosure, a movement confirmation system is provided including: an acquisition unit configured to acquire a target image in which a target region is imaged; a generation unit configured to generate a synthetic image in which a smoke image is synthesized with the target image acquired by the acquisition unit; and a determination unit configured to determine whether a smoke detection device which is provided in the target region and which determines whether smoke is being generated in the target region on the basis of an image is operating correctly using the synthetic image.

[0033] According to a twenty-fourth aspect of the present disclosure, an image generation method is provided that is performed by an image generation device which is a computer, the image generation method including: causing an acquisition unit to acquire a target image in which a target region is imaged; causing a generation unit to generate a synthetic image in which a smoke image is synthesized with the target image acquired by the acquisition unit; and determining whether a smoke detection device which is provided in the target region and which determines whether smoke is being generated in the target region on the basis of an image is operating correctly using the synthetic image.

[0034] According to a twenty-fifth aspect of the present disclosure, a program is provided causing an image generation device which is a computer to perform: acquiring a target image in which a target region is imaged; generating a synthetic image in which a smoke image is synthesized with the acquired target image; and determining whether a smoke detection device which is provided in the target region and which determines whether smoke is being generated in the target region on the basis of an image is operating correctly using the synthetic image. Advantageous Effects of Invention

[0035] According to the present disclosure, it is possible to provide a flame detection device, a flame detection method, an image generation device, a movement confirmation system, an image generation method, and a program in a flame detection system and a flame detection system that can detect a flame with high accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0036] [FIG. 1] A diagram illustrating an example of a configuration of a flame detection system to which a flame detection device according to a first embodiment of the present disclosure is applied. [FIG. 2] A block diagram illustrating an example of a configuration of the flame detection device according to the first embodiment of the present disclosure. [FIG. 3A] A diagram illustrating a process that is performed by a flame candidate region extracting unit according to the first embodiment of the present disclosure. [FIG. 3B] A diagram illustrating a process that is performed by the flame candidate region extracting unit according to the first embodiment of the present disclosure. [FIG. 4] A diagram illustrating a process that is performed by a flame determining unit according to the first embodiment of the present disclosure. [FIG. 5] A diagram illustrating a process that is performed by the flame determining unit according to the first embodiment of the present disclosure. [FIG. 6] A diagram illustrating a process that is performed by the flame determining unit according to the first embodiment of the present disclosure. [FIG. 7] A diagram illustrating a process that is performed by the flame determining unit according to the first embodiment of the present disclosure. [FIG. 8] A diagram illustrating a process that is performed by the flame determining unit according to the first embodiment of the present disclosure. [FIG. 9] A flowchart illustrating a process flow that is performed by the flame detection device according to the first embodiment of the present disclosure. [FIG. 10] A diagram illustrating a process that is performed by the flame determining unit according to the first embodiment of the present disclosure. [FIG. 11] A diagram illustrating a process that is performed by the flame determining unit according to the first embodiment of the present disclosure. [FIG. 12] A diagram illustrating an example of a configuration of a flame detection system to which a flame detection device according to a second embodiment of the present disclosure is applied. [FIG. 13] A block diagram illustrating an example of a configuration of the flame detection device according to the second embodiment of the present disclosure. [FIG. 14A] A diagram illustrating a process that is performed by a flame determining unit according to the second embodiment of the present disclosure. [FIG. 14B] A diagram illustrating a process that is performed by a flame determining unit according to the second embodiment of the present disclosure. [FIG. 14C] A diagram illustrating a process that is performed by a flame determining unit according to the second embodiment of the present disclosure. [FIG. 15] A flowchart illustrating a process flow that is performed by the flame detection device according to the second embodiment of the present disclosure. [FIG. 16] A flowchart illustrating a process flow that is performed by the flame detection device according to the second embodiment of the present disclosure. [FIG. 17] A flowchart illustrating a process flow that is performed by the flame detection device according to the second embodiment of the present disclosure. [FIG. 18] A diagram illustrating an example of a target image, a flame candidate region, a statistic, and an index according to the second embodiment of the present disclosure. [FIG. 19] A diagram illustrating an example of a target image, a flame candidate region, a statistic, and an index according to the second embodiment of the present disclosure. [FIG. 20] A diagram illustrating an example of a target image, a flame candidate region, a statistic, and an index according to the second embodiment of the present disclosure. [FIG. 21] A diagram illustrating an example of a target image, a flame candidate region, a statistic, and an index according to the second embodiment of the present disclosure. [FIG. 22] A diagram illustrating an example of an index when a target image is a moving image according to the second embodiment of the present disclosure. [FIG. 23] A diagram illustrating an example of a configuration of a flame detection system to which an image generation device according to a third embodiment of the present disclosure is applied. [FIG. 24] A block diagram illustrating an example of a configuration of the image generation device according to the third embodiment of the present disclosure. [FIG. 25] A block diagram illustrating an example of a configuration of an operation control device according to the third embodiment of the present disclosure. [FIG. 26] A diagram illustrating an example of a target image that is displayed by the image generation device according to the third embodiment of the present disclosure. [FIG. 27] A diagram illustrating an example of a synthetic image that is displayed by the image generation device according to the third embodiment of the present disclosure. [FIG. 28] A flowchart illustrating a process flow that is performed by the image generation device according to the third embodiment of the present disclosure. [FIG. 291 A diagram illustrating a marker according to Modified Example 1 of the third embodiment of the present disclosure. [FIG. 30] A diagram illustrating an example of a target image that is displayed by the image generation device according to Modified Example 1 of the third embodiment of the present disclosure. [FIG. 31 A] A diagram illustrating an example of a synthetic image that is displayed by the image generation device according to Modified Example 1 of the third embodiment of the present disclosure. [FIG. 31B] A diagram illustrating an example of a synthetic image that is displayed by the image generation device according to Modified Example 1 of the third embodiment of the present disclosure. DESCRIPTION OF EMBODIMENTS

[0037] (First embodiment) Hereinafter, a first embodiment of the present disclosure will be described with reference to the accompanying drawings.

[0038] FIG. 1 is a diagram illustrating an example of a configuration of a flame detection system 1 to which a flame detection device 10 according to this embodiment is applied. For example, the flame detection system 1 includes a camera CA and a flame detection device 10. The camera CA and the flame detection device 10 are communicatively connected by short-range communication using a communication network or a wireless local area network (LAN) or via a universal serial bus (USB) cable or the like. The camera CA and the flame detection device 10 may be provided as a unified body. That is, the camera CA may be incorporated into the flame detection device 10. A monitoring target T is a region or an object which is monitored by the flame detection system 1. The monitoring target T may be arbitrarily set. For example, the monitoring target T is a roadway, tunnel premises, a building, or building premises. The camera CA images the monitoring target T. The camera CA may capture a moving image or may capture a still image. The camera CA outputs a captured image to the flame detection device 10. For example, the flame detection device 10 is a computer and is realized by a personal computer (PC), a server device, or the like. The flame detection device 10 acquires an image (a target image) in which the monitoring target T is imaged from the camera CA and determines whether a flame is shown in the acquired image.

[0039] FIG. 2 is a block diagram illustrating an example of the configuration of the flame detection device 10 according to this embodiment. For example, the flame detection device 10 includes an image acquiring unit 100, a flame candidate region extracting unit 101, a flame determining unit 102, a determination result output unit 103, and an image information storage unit 104. The functional units (the image acquiring unit 100, the flame candidate region extracting unit 101, the flame determining unit 102, and the determination result output unit 103) included in the flame detection device 10 are realized by causing a central processing unit (CPU) provided as hardware in the flame detection device 10 to execute a program.

[0040] The image acquiring unit 100 acquires image information of a target image. The image acquiring unit 100 acquires image information of a target image captured by the camera CA. The image acquiring unit 100 outputs the acquired image information to the flame candidate region extracting unit 101.

[0041] The flame candidate region extracting unit 101 extracts a flame candidate region from the target image. A flame candidate region is a region in which an object like a flame is shown in the target image. A specific method by which the flame candidate region extracting unit 101 extracts a flame candidate region from the target image will be described later in detail.

[0042] The flame determining unit 102 determines whether a flame is shown in the flame candidate region. The flame determining unit 102 determines whether a flame is shown in the flame candidate region on the basis of movement of an object imaged in the flame candidate region. A specific method by which the flame determining unit 102 determines whether a flame is shown in the flame candidate region will be described later in detail.

[0043] The determination result output unit 103 outputs a determination result from the flame determining unit 102, that is, whether a flame is shown in the target image. For example, the determination result output unit 103 may be configured to display the determination result by displaying the determination result on a display which is not illustrated. In this case, when a flame is shown in the target image, a user’s attention may be attracted by displaying a message indicating that there is a likelihood of a fire, outputting an alarm sound, or the like.

[0044] The image information storage unit 104 stores image information. Image information is image information of a target image. The image information may include information indicating the flame candidate region extracted by the flame candidate region extracting unit 101, the determination result from the flame determining unit 102, or information indicating an index or the like used for the determination. For example, the image information storage unit 104 is constituted by a storage medium such as a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), or a read only memory (ROM) or an arbitrary combination of such storage media.

[0045] A method by which the flame candidate region extracting unit 101 extracts a flame candidate region from a target image will be described below. The flame candidate region extracting unit 101 extracts a flame candidate region from a target image through a plurality of step-by-step processes.

[0046] In a first step, the flame candidate region extracting unit 101 extracts a first candidate region on the basis of color information in the target image. Specifically, the flame candidate region extracting unit 101 digitalizes pixels in the target image according to whether each pixel value shows a color likely to be a flame. For example, when the color information is expressed by red, green, and blue (RGB), the color likely to be a flame is a color in which an R value of a red component in the color information is greater than a threshold value or a color in which a difference (R-G) value between the red component and a green component is greater than a threshold value. For example, when the color information is expressed by hue, saturation, and value brightness (HSV), the color likely to be a flame is a color in which an H value of a hue component is close to red and a V value of a brightness component is greater than a threshold value. The flame candidate region extracting unit 101 extracts a region including a pixel group indicating a color likely to be a flame out of the digitized pixels and sets the extracted region as the first candidate region. In this case, the flame candidate region extracting unit 101 may exclude a region of which an area is less than a threshold value as noise out of the regions including the pixel groups showing the color likely to be a flame and set a region of which the area is equal to or greater than the threshold value as the first candidate region.

[0047] In a second step, the flame candidate region extracting unit 101 performs a tracking process of determining whether a first candidate region extracted from a target image (hereinafter referred to as a second image) which is captured after the target image (hereinafter referred to as a first image) from which the first candidate region has been extracted is based on the same object.

[0048] FIG. 3 (FIGS. 3A and 3B) is a diagram illustrating the tracking process. FIG. 3A schematically illustrates an example of a target image TGI. In the target image TGI, regions RI and R2 are extracted as the first candidate region. For example, a blinking red light is shown in the region RI. A flame is shown in the region R2. Such a red light or a flame exhibits a color likely to be a flame and thus is extracted as the first candidate region. The flame candidate region extracting unit 101 sets a pixel located at a specific position, for example, a pixel located at the center of gravity, in the region extracted as the first candidate region as a tracking point and acquires positional coordinates of the tracking point. In the example illustrated in the drawing, the flame candidate region extracting unit 101 acquires positional coordinates of a pixel RIG as the tracking point of the region R1 and a pixel R2G as the tracing point of the region R2.

[0049] FIG. 3B schematically illustrates an example of a target image TG2. The target image TG2 is a target image which is captured at a time later in a time series than the target image TGI. For example, in a moving image, the target image TG2 is an image captured in a frame next to the target image TGI. In the target image TG2, a region R2# is extracted as the first candidate region. The flame candidate region extracting unit 101 acquires positional coordinates of a pixel R2G# as the tracking point of the region R2# extracted as the first candidate region. The flame candidate region extracting unit 101 calculates the distance between the tracking point (the pixel R2G) acquired in the first image and the tracking point (the pixel R2G#) acquired in the second image. When the calculated distance is less than a threshold value, the flame candidate region extracting unit 101 determines that the first candidate region extracted in the first image and the first candidate region extracted in the second image are based on the same object. In this case, the tracking process is performed on a target image captured after the second image using the region R2 as a tracking target. The method of setting the threshold value may be appropriately determined and may be set, for example, with respect to an object moving at the highest speed in the monitoring target T. For example, when the monitoring target T is a roadway or tunnel premises, the threshold value may be set on the basis of a distance in which a vehicle traveling at a highest speed of a vehicle passing through the roadway or the tunnel premises or at a speed obtained by adding a predetermined margin to the highest speed moves in a time interval in which the target image TGI and the target image TG2 are captured. On the other hand, when a first candidate region is not extracted in the vicinity of the tracking point (the pixel RIG) acquired in the first image in the second image, the flame candidate region extracting unit 101 determines that the first candidate region extracted in the first image and the first candidate region extracted in the second image are not based on the same object. That is, it is determined that the object imaged in the region RI of the first image cannot be tracked in the second image. In this case, the region RI is excluded from the tracking target, and the tracking process is not performed on the target image captured after the second image.

[0050] The flame candidate region extracting unit 101 determines whether the first candidate region is to be set as a second candidate region on the basis of the result of the tracking process. When the first candidate region extracted in the first step is a tracking target, that is, when the region R2# is extracted in the vicinity of the region R2 extracted in the first image in the second image, the flame candidate region extracting unit 101 sets the region R2 and / or the region R2# as the second candidate region. On the other hand, when the first candidate region extracted in the first step is excluded from the tracking target, that is, when the first candidate region is not extracted in the vicinity of the region RI extracted in the first image in the second image, the flame candidate region extracting unit 101 does not set the first candidate region (the region RI) as the second candidate region. In this way, by performing the tracking process in the second step, the second candidate region can be set even when an object moving more likely to be a flame is moving.

[0051] Tn this case, the flame candidate region extracting unit 101 may be configured to set a region which can be tracked in a predetermined number or greater of target images which are successive in a time series, for example, images corresponding to several frames, as the second candidate region and not to set a region which cannot be tracked in images corresponding to several frames as the second candidate region.

[0052] Tn a third step, the flame candidate region extracting unit 101 calculates a degree to which the second candidate region is likely to be a flame on the basis of shape complexity in the second candidate region and / or a color change in a time series.

[0053] First, a method by which the flame candidate region extracting unit 101 calculates the degree to which the second candidate region is likely to be a flame on the basis of shape complexity in the second candidate region will be described below. The flame candidate region extracting unit 101 extracts a numerical progression Pn indicating coordinates of a pixel group constituting a contour as contour information. For example, the numerical progression Pn is a numerical progression in which pixel groups constituting the contour are sequentially arranged in a predetermined direction, for example, in a clockwise direction or a counterclockwise direction. The flame candidate region extracting unit 101 selects three pixels which are separated by a predetermined distance from the numerical progression Pn. Specifically, the flame candidate region extracting unit 101 selects there pixels (Pk-d, Pk, and Pk+d) separated by a distance d from the numerical progression Pn {Pl, P2,..., Pn}.

[0054] The flame candidate region extracting unit 101 calculates an angle formed by line segments which are formed by connecting the selected three pixels along the contour. Specifically, the flame candidate region extracting unit 101 calculates vectors VI and V2 which are directed from a start point Pk to other points (Pk-d) and (Pk+d) in the coordinates of the three pixels (Pk-d, Pk, and Pk+d) selected from the numerical progression Pn. The vector V1 is a vector directed from the pixel (Pk) to the pixel (Pk-d). The vector V2 is a vector directed from the pixel (Pk) to the pixel (Pk+d). For example, the flame candidate region extracting unit 101 calculates an angle 0 formed by line segments using an inner product as represented by Expression (1).

[0055] dxl*dx2+dyl*dy2 =Mdxl A2+dyl A2)xMdx2A2+dy2A2)xcosO ...(1) Here, (dxl, dyl): coordinate values of vector V1 (dx2, dy2): coordinate values of vector V2 0: angle formed by line segments

[0056] In this way, the flame candidate region extracting unit 101 calculates an angle formed by line segments connecting three pixels selected from a pixel group constituting a contour of a flame candidate region R along the contour as an angle formed by the contour line. For example, the flame candidate region extracting unit 101 calculates angles formed by line segments with each of all the pixels constituting the contour as a start point as the angle formed by the contour line. The flame candidate region extracting unit 101 calculates a degree of being like a flame based on shape complexity using a statistic of the angles formed by the contour lines. For example, the static may be arbitrarily determined as long as it is a statistic serving as an index indicating whether the angles formed by the contour line are concentrated on a specific angle such as 180°. For example, an average value, a representative value, a mode, a maximum value, a minimum value, a variance, a standard deviation, and a combination thereof can be used as the statistic.

[0057] A specific method by which the flame candidate region extracting unit 101 calculates an index using the statistic will be described below. In this embodiment, two indices including (1) a degree of concentration and (2) a deviation of angles which are modes are used as the index indicating shape complexity of a flame candidate region. (1) The degree of concentration is an index indicating a degree to which the angles formed by the contour line are concentrated on a specific angle and is a ratio of the number of angles which are modes to the total number of angles formed by the contour lines. For example, when line segments with each of all the pixels constituting the contour as a start point are generated and angles formed by the generated line segments are acquired, the total number of angles formed by the contour line is the same as the total number of pixels constituting the contour line. In this case, when the degree of concentration calculated from the second candidate region is less than a threshold value, the flame candidate region extracting unit 101 determines that an object imaged in the second candidate region is likely to be a flame. On the other hand, when the degree of concentration calculated from the second candidate region is equal to or greater than the threshold value, the flame candidate region extracting unit 101 determines that the object imaged in the second candidate region is not likely to be a flame.

[0058] (2) The deviation of angles which are modes is an index indicating a degree to which the modes of the angles formed by the contour line in the images are not concentrated, that is, a degree of deviation, when a moving image obtained by capturing the monitoring target T is used. For example, the flame candidate region extracting unit 101 calculates a mode in each target image constituting a moving image. Here, the mode is an angle which is the mode out of the angles formed by the contour line in the flame candidate region extracted from the target image. When a deviation of the modes calculated from the second candidate regions which are successive in a time series is large and equal to or greater than a threshold value, the flame candidate region extracting unit 101 determines that an object imaged in the second candidate regions is likely to be a flame. On the other hand, when the deviation of the modes calculated from the second candidate regions is small and equal to or less than the threshold value, the flame candidate region extracting unit 101 determines that the object imaged in the second candidate regions is not likely to be a flame.

[0059] A method by which the flame candidate region extracting unit 101 calculates a degree to which the second candidate region is likely to be a flame on the basis of a color change in a time series in the second candidate region will be described below. For example, the flame candidate region extracting unit 101 acquires pixel values of the pixel groups constituting the second candidate region shown in each of a plurality of target images which are captured in a time series. The flame candidate region extracting unit 101 calculates a rhythm of change of a pixel value (frequency characteristics), for example, by performing frequency analysis, for example, fast Fourier transform (FFT), on a time-series change of a representative pixel value selected from the acquired pixel values, for example, an average value thereof. The flame candidate region extracting unit 101 calculates a degree to which the rhythm of change of a pixel value (frequency characteristics) is similar to a rhythm of change of a flame color as the degree to which the second candidate region is likely to be a flame.

[0060] FIG. 4 is a diagram illustrating the method of calculating the degree of being like a flame in each pixel constituting a second candidate region. Frequency characteristics calculated by performing FFT on the pixel values are illustrated in FIG. 4. The horizontal axis of the graph illustrated in FIG. 4 represents the frequency, and the vertical axis represents the signal intensity.

[0061] In the graph illustrated in the upper part of FIG. 4, frequency characteristics of the pixel values in a region including a flame in a moving image are represented. In this drawing, the spectrum (signal intensity) in a range of 1.0 [Hz] <frequency f <4.5 [Hz] is the largest. This represents that, when a situation in which a flame blazes is imaged as a moving image, the color changes in a rhythm of about 1 to 4.5 times per second.

[0062] In the graph illustrated in the lower part of FIG. 4, frequency characteristics of pixel values in a region including a situation in which a vehicle passes in the moving image are illustrated. Tn this drawing, the spectrum (signal intensity) in a frequency range of equal to or less than 1.5 [Hz] is the largest. This represents that, when the situation in which a vehicle passes is imaged as a moving image, the color changes in a rhythm of equal to or less than about 1.5 times per second.

[0063] For example, when the frequency characteristics of the pixels based on the pixel groups constituting the second candidate region have a peak, that is, the largest signal intensity, in a predetermined frequency range of 1.0 [Hz] <frequency f <4.5 [Hz], the flame candidate region extracting unit 101 determines that the object imaged in the second candidate region is likely to be a flame. When the frequency characteristics of the pixels based on the pixel groups constituting the second candidate region do not have a peak in the predetermined frequency, the flame candidate region extracting unit 101 determines that the object imaged in the second candidate region is not likely to be a flame. The flame candidate region extracting unit 101 sets a second candidate region in which the degree of being like a flame is the largest and an object imaged in the second candidate region is determined to be a flame as a flame candidate region.

[0064] A method by which the flame determining unit 102 determines whether a flame is shown in a flame candidate region will be described below. The flame determining unit 102 determines whether a flame is shown in the flame candidate region on the basis of movement of an object imaged in the flame candidate region.

[0065] The flame determining unit 102 divides the flame candidate region into a lattice shape and sets a tracking point in each sub area. The flame determining unit 102 calculates an optical flow at each tracking point. For example, an optical flow is a concept indicating movement of an object imaged in an image and is a concept indicating a moving direction of a flame imaged in the image. Here, the flame determining unit 102 acquires a color change at each tracking point set in each sub area obtained by dividing the flame candidate region into a lattice shape. The flame determining unit 102 performs an arithmetic operation on the color change at each tracking point. For example, when a color changes from a first point to a second point, the flame determining unit 102 calculates a vector directed from the first point to the second point. The flame determining unit 102 calculates a plurality of vectors by performing this arithmetic operation on the plurality of tracking points. The flame determining unit 102 calculates a vector obtained by combining the calculated plurality of vectors as an optical flow indicating a direction and a magnitude in which a flame moves.

[0066] The flame determining unit 102 calculates an index indicating whether an object imaged in a flame candidate region moves like a flame on the basis of a time-series change of the optical flow.

[0067] In general, a flame adopts a behavior blazing slowly while repeating movement of enlargement or reduction, that is, fluctuation. On the other hand, a moving object such as a vehicle generally moves at a constant speed or while changing a speed in a constant direction. By employing this movement specific to a flame, the flame determining unit 102 in this embodiment calculates an index indicating whether an object imaged in a flame candidate region moves like a flame on the basis of the time-series change of the optical flow.

[0068] FIG. 5 is a diagram illustrating an example of an optical flow that is calculated from a moving image in which a flame is imaged. In FIG. 5, frame images from time t=0 to time t=ll are illustrated in a time series. A situation in which a flame shakes is imaged in each frame image. An optical flow which is calculated in a flame candidate region (a region including a flame) extracted from each frame image is indicated by an arrow. As illustrated in the example of this drawing, when a flame is imaged, the optical flow calculated in the frame images exhibits various types of movement and does not move in a constant direction unlike a moving object such as a vehicle.

[0069] The flame determining unit 102 extracts a value in an x-axis component (hereinafter referred to as an x component value) and a value of a y-axis component (hereinafter referred to as a y component value) in the optical flow calculated for each tracking point in a time series. The flame determining unit 102 calculates a simple weighted average value of the x component values (hereinafter referred to as an x average value) in a time interval on the basis of a time-series change of the x component value in the optical flow. The flame determining unit 102 calculates a simple weighted average value of the y component values (hereinafter referred to as a y average value) in a time interval on the basis of a time-series change of the y component value in the optical flow.

[0070] The x-axis direction and the y-axis direction may be arbitrarily determined. For example, the x-axis direction is a horizontal direction (a lateral direction) in a target image, and the y-axis direction is a vertical direction (a longitudinal direction) in the target image. For example, a direction of a first main component may be defined as an x axis and a direction of a second main component may be defined as a y axis, where the first main component and the second main component are acquired by performing main component analysis on a horizontal component and a vertical component in an image in the optical flow. Alternatively, when a road is included in the monitoring target T, a road passing direction may be defined as the x-axis direction, and a direction perpendicular to the road passing direction may be defined as the y-axis direction.

[0071] FIG. 6 is a diagram illustrating an example of a time-series change of an x component value and an x average value in the optical flow. In FIG. 6, the horizontal axis represents a time, and the vertical axis represents a component value. As illustrated in the example of this drawing, when movement of enlargement or reduction is repeated like a flame, the x component value repeatedly moves above or below the x average value.

[0072] The flame determining unit 102 calculates a degree of fluctuation of the x component value with respect to the x average value as an index indicating whether an object moves like a flame. Here, fluctuation of the x component value with respect to the x average value means that the x component value crosses the x average value. More specifically, the x component value changes from less than the x average value to equal to or greater than the x average value, or the x component value changes from equal to or greater than the x average value to less than the x average value. A number of times at which the x component value crosses the x average value (hereinafter referred to as a number of crossings) can be used as the degree of fluctuation.

[0073] Similarly to the y direction, the flame determining unit 102 calculates a degree of fluctuation of the y component value with respect to the y average value as an index indicating whether an object moves like a flame. Here, fluctuation of the y component value with respect to the y average value means that the y component value crosses the y average value. More specifically, the y component value changes from less than the y average value to equal to or greater than the y average value, or the y component value changes from equal to or greater than the y average value to less than the y average value. A number of times at which the y component value crosses the y average value (hereinafter referred to as a number of crossings) can be used as the degree of fluctuation. Hysteresis may be provided to determination of whether a component value crosses an average value. For example, when the x component value changes from a value less than the x average value to a value equal to or greater than (x average value + predetermined value a) or the x component value changes from a value equal to or greater than the x average value to a value less than (x average value - predetermined value p), it may be determined that the component value crosses the average value. Here, the predetermined values a and P may be the same or different.

[0074] FIG. 7 is a diagram illustrating an example of time-series changes of the x component value and the y component value in an optical flow of a flame candidate region in which a flame is imaged. In FIG. 7, the horizontal axis represents a time, and the vertical axis represents a component value. Avex indicates the x average value. In the example illustrated in this drawing, the number of crossings in the x direction is 8, and the number of crossings in the y direction is 6. A sum number of crossings of the number of crossings in the x direction and the number of crossings in the y direction is 14.

[0075] FIG. 8 is a diagram illustrating an example of time-series changes of the x component value and the y component value in an optical flow of a flame candidate region in which a moving vehicle is imaged. In FIG. 8, the horizontal axis represents a time, and the vertical axis represents a component value. Avex indicates the x average value. Avey indicates the y average value. In the example illustrated in this drawing, the number of crossings in the x direction is 3, and the number of crossings in the y direction is 2. The total number of crossings of the number of crossings in the x direction and the number of crossings in the y direction is 5.

[0076] The flame determining unit 102 calculates the number of crossings as a degree of fluctuation, that is, an index indicating whether an object moves like a flame. For example, when the number of crossings calculated from the flame candidate region is equal to or greater than a threshold value, the flame determining unit 102 determines that a flame is shown in the flame candidate region. On the other hand, when the number of crossings calculated from the flame candidate region is less than the threshold value, the flame determining unit 102 determines that a flame is not shown in the flame candidate region.

[0077] The flame determining unit 102 may determine whether a flame is shown in the flame candidate region using only one of the number of crossings in the x direction and the number of crossings in the y direction. The flame determining unit 102 may determine whether a flame is shown in the flame candidate region using only one of a number of times (a first number of crossings) at which the x component value changes from less than the x average value to equal to or greater than the x average value and a number of times (a second number of crossings) at which the x component value changes from equal to or greater than the x average value to less than the x average value.

[0078] A process flow that is performed by the flame detection device 10 will be described below with reference to FIG. 9. FIG. 9 is a flowchart illustrating a process flow that is performed by the flame detection device 10 according to this embodiment.

[0079] The flame detection device 10 acquires target images (frame images) which are successive in a time series (Step S10). For example, the flame detection device 10 acquires a moving image in which a monitoring target T is imaged by the camera CA as the target images which are successive in a time series. Then, the flame detection device 10 extracts a first candidate region from each target image (Step Sil). The flame detection device 10 extracts a region with a color like a flame as the first candidate region on the basis of color information in each target image. Then, the flame detection device 10 determines whether the first candidate region is used as a second candidate region on the basis of a result of tracking of the first candidate region (Step S12). When the first candidate region extracted in a certain frame image can be tracked over several frames subsequent to a next frame, the flame detection device 10 sets the first candidate region as a second candidate region. On the other hand, when the first candidate region extracted in a certain frame image cannot be tracked over several frames subsequent to a next frame, the flame detection device 10 does not set the first candidate region as a second candidate region.

[0080] Then, the flame detection device 10 determines whether the second candidate region is set as a flame candidate region on the basis of features, that is, shape complexity of a contour and / or a time-series change of a color, of the second candidate region (Step SI3). For example, when angles formed by the contour shape of the second candidate region are not concentrated on a specific angle, the flame detection device 10 determines the second candidate region to be a flame candidate region. On the other hand, when angles formed by the contour shape of the second candidate region are concentrated on a specific angle, the flame detection device 10 does not determine the second candidate region to be a flame candidate region. Alternatively, when the mode of the angles formed by the contour shape of the second candidate region deviates in a time series, the flame detection device 10 determines the second candidate region to be a flame candidate region. On the other hand, when the mode of the angles formed by the contour shape of the second candidate region does not deviate in a time series, the flame detection device 10 does not determine the second candidate region to be a flame candidate region. For example, the flame detection device 10 calculates frequency characteristics of pixel values in the second candidate region and determines the second candidate region as a flame candidate region when the spectrum of the pixel values is similar to the spectrum of a flame. On the other hand, when the spectrum of the pixel values in the second candidate region is not similar to a spectrum of a flame, the flame detection device 10 does not determine the second candidate region as a flame candidate region.

[0081] Then, the flame detection device 10 calculates an optical flow in the flame candidate region (Step S14). The flame detection device 10 calculates the optical flow by calculating a vector indicating a motion of each of a plurality of tracking points set in the flame candidate region and combining the calculated vectors.

[0082] Then, the flame detection device 10 determines whether a flame is shown in the flame candidate region on the basis of a degree of fluctuation of a component of the optical flow with respect to an average value (a number of crossings) (Step SI5). The flame detection device 10 calculates the number of crossings for each of the x component and the y component of the optical flow and sets a sum value of the number of crossings of the x component and the number of crossings of the y component as the degree of fluctuation (the number of crossings). When the number of crossings is equal to or greater than a threshold value, the flame detection device 10 determines that a flame is shown in the flame candidate region. On the other hand, when the number of crossings is less than the threshold value, the flame detection device 10 determines that a flame is not shown in the flame candidate region.

[0083] As described above, the flame detection device 10 according to this embodiment includes the image acquiring unit 100, the flame candidate region extracting unit 101, and the flame determining unit 102. The image acquiring unit 100 acquires a moving image. The moving image includes a plurality of target images in which a monitoring target T is captured in a time series. The flame candidate region extracting unit 101 extracts a flame candidate region from each target image. The flame determining unit 102 calculates an index indicating that an object shown in the flame candidate region moves like a flame. The flame determining unit 102 calculates the index using a degree of fluctuation of an amount of movement of a tracking point set in the flame candidate region with respect to an average value of the amount of movement. The flame determining unit 102 determines whether a flame is shown in the flame candidate region on the basis of the calculated index.

[0084] Accordingly, the flame detection device 10 according to this embodiment can determine whether an object imaged in the flame candidate region is a flame according to whether the object shown in the flame candidate region moves like a flame. As a result, it is possible to accurately determine whether a flame is shown in an image even when a 36 fire breaks out in a moving object.

[0085] In the flame detection device 10 according to this embodiment, the flame determining unit 102 calculates an optical flow of a pixel included in the flame candidate region as the amount of movement (for example, an x component value and a y component value). Accordingly, with the flame detection device 10 according to this embodiment, it is possible to quantitatively calculate movement of an object imaged in the flame candidate region using the optical flow.

[0086] In the flame detection device 10 according to this embodiment, the flame determining unit 102 calculates the index using at least one of a fust number of crossings at which the amount of movement changes from a value less than the average value to a value greater than the average value and a second number of crossings at which the amount of movement changes from a value greater than the average value to a value less than the average value. Accordingly, with the flame detection device 10 according to this embodiment, it is possible to easily determine whether an object imaged in the flame candidate region moves like a flame using the number of crossings.

[0087] In the flame detection device 10 according to this embodiment, the flame determining unit 102 calculates an amount of movement and an average value in each of the x-axis direction (a first direction) and the y-axis direction (a second direction) in the target images. The flame determining unit 102 determines whether a flame is shown in the flame candidate region on the basis of the calculated number of crossings (index) in each direction. Accordingly, with the flame detection device 10 according to this embodiment, it is possible to accurately determine whether a flame is shown in the flame candidate region on the basis of fluctuation of movement in each direction using vectors indicating movement, for example, each component in the optical flow.

[0088] In the flame detection device 10 according to this embodiment, the flame candidate region extracting unit 101 extracts a first candidate region on the basis of color information included in a plurality of target images. The flame candidate region extracting unit 101 determines whether the first candidate region extracted from the first image and a first candidate region extracted from a second image captured after the first image are based on the same object. When the first candidate region extracted from the first image and a first candidate region extracted from the second image are based on the same object, the flame candidate region extracting unit 101 determines the first candidate region extracted from the first image and / or the first candidate region extracted from the second image as a second candidate region. The flame candidate region extracting unit 101 determines the flame candidate region from the second candidate region. In the flame detection device 10 according to this embodiment, the flame candidate region extracting unit 101 calculates a degree to which the second candidate region is likely to be a flame on the basis of frequency characteristics in which a color in the second candidate region changes and determines whether the second candidate region is set as the flame candidate region on the basis of the calculated degree of being like a flame. Accordingly, in the flame detection device 10 according to this embodiment, a region in which a color or a change of the color is likely to be a flame can be determined as a flame candidate region on the basis of the color of an object imaged in the target image and a rhythm in which the color changes.

[0089] Modified Example 1 of the flame detection system according to this embodiment will be described below. This modified example is different from the aforementioned embodiment in that a regression line is used in the average value (the x average value and the y average value). The regression line is a straight line most fitted to a distribution of the component (the x component value or the y component value). An arbitrary method in the related art can be used as a method of calculating the regression line. For example, a least square method can be used as a method of optimally calculating a coefficient and an intercept in the regression line.

[0090] FTG. 10 is a diagram illustrating a process that is performed by the flame determining unit 102 according to Modified Example 1 of this embodiment. In the example illustrated in FIG. 10, a time-series change of the x component value in an optical flow and a regression line of the x component value are used as the x average value. As illustrated in the example of this drawing, the x component value may decrease gradually while fluctuating vertically. In this case, when the number of crossings is calculated as a simple weighted average value, it is difficult to appropriately acquire a degree of fluctuation. As a countermeasure, in this modified example, the regression line of the x component is used as the x average value. Accordingly, even when the x component value decreases gradually (or increases gradually) while fluctuating vertically, it is possible to appropriately calculate the degree of fluctuation of the x component value.

[0091] Modified Example 2 of the flame detection system according to this embodiment will be described below. This modified example is different from the aforementioned embodiment in that a simple weighted average value of pre-and-post values (hereinafter referred to as a pre-and-post average value) is used as the average value (the x average value and the y average value). The pre-and-post average value is a simple weighted average value of the pre-and-post values of the component value (the x component value or the y component value). For example, the pre-and-post average value is a simple weighted average value of the pre-and-post x component values in a tinier series with respect to the time at which the average value is calculated.

[0092] FIG. 11 is a diagram illustrating a process that is performed by the flame determining unit 102 according to Modified Example 2 of this embodiment. In the example illustrated in FIG. 11, a time-series change of the x component values in the optical flow and a pre-and-post average value of the x component values are used as the x average value. As illustrated in the example of this drawing, while the x component value fluctuates vertically and decreases gradually, a decreasing rate thereof may not be constant. In this case, when the number of crossings is calculated on the basis of the simple weighted average value or the regression line, it is difficult to appropriately acquire the degree of fluctuation. As a countermeasure, in this modified example, the pre-and-post average value of the x component values is used as the x average value. Accordingly, even when the decreasing rate is not constant while the x component value fluctuates vertically and decreases gradually (or increases gradually), the x average value can be set on the basis of the x component values in the vicinity thereof, and thus it is possible to appropriately calculate a degree to which the x component value fluctuates.

[0093] In the aforementioned embodiment, an example of the method for the tracking process has been described, and the present disclosure is not limited thereto. Specifically, when a distance between a tracking point (the pixel R2G) acquired in the first image illustrated in FIG. 3 (FIGS. 3A and 3B) and a tracking point (the pixel R2G#) acquired in the second image is less than a threshold value, the flame candidate region extracting unit determines that the first candidate region extracted in the first image and the first candidate region extracted in the second image are based on the same object, but determination of whether the region R2 and the region R2# are based on the same object is not limited thereto. For example, when the tracking point (the pixel R2G) of the region R2 is included in the region R2#, the flame candidate region extracting unit may determine that the first candidate region extracted in the first image and the first candidate region extracted in the second image are based on the same object. When the tracking point (the pixel R2G#) of the region R2# is included in the region R2, the flame candidate region extracting unit may determine that the first candidate region extracted in the first image and the first candidate region extracted in the second image are based on the same object.

[0094] (Second embodiment) A second embodiment of the present disclosure will be described below with reference to the accompanying drawings.

[0095] FIG. 12 is a diagram illustrating an example of a configuration of a flame detection system 1A to which a flame detection device 10A according to this embodiment is applied. For example, the flame detection system 1A includes a camera CAA and a flame detection device 10A. The camera CAA and the flame detection device 10A are communicatively connected by short-range communication using a communication network or a wireless local area network (LAN) or via a universal serial bus (USB) cable or the like. A monitoring target TA is a region or an object which is monitored by the flame detection system 1 A. The monitoring target TA may be arbitrarily set. For example, the monitoring target TA is a roadway, tunnel premises, a building, or building premises. The camera CAA images the monitoring target TA. The camera CAA may capture a moving image or may capture a still image. The camera CAA outputs a captured image to the flame detection device 10A. For example, the flame detection device 10A is a computer and is realized by a personal computer (PC), a server device, or the like. The flame detection device 10A acquires an image (a target image) in which the monitoring target TA is imaged from the camera CAA and determines whether a flame is shown in the acquired image.

[0096] FIG. 13 is a block diagram illustrating an example of the configuration of the flame detection device 10A according to this embodiment. For example, the flame detection device 10A includes an image acquiring unit 100A, a flame candidate region extracting unit 101A, a flame determining unit 102A, a determination result output unit 103 A, and an image information storage unit 104A. The functional units (the image acquiring unit 100A, the flame candidate region extracting unit 101 A, the flame determining unit 102A, and the determination result output unit 103A) included in the flame detection device 10A are realized by causing a central processing unit (CPU) provided as hardware in the flame detection device lOAto execute a program.

[0097] The image acquiring unit 100A acquires image information of a target image. The image acquiring unit 100A acquires image information of a target image acquired by the camera CAA. The image acquiring unit 100A outputs the acquired image information to the flame candidate region extracting unit 101 A.

[0098] The flame candidate region extracting unit 101A extracts a flame candidate region from the target image. A flame candidate region is a region in which an object like a flame is shown in the target image. The flame candidate region extracting unit 101A extracts a flame candidate region on the basis of color information in a target image. Specifically, the flame candidate region extracting unit 101 digitalizes pixels in the target image according to whether each pixel value indicates a color likely to be a flame. For example, when the color information is expressed by red, green, and blue (RGB), the color likely to be a flame is a color in which an R value of a red component in the color information is greater than a threshold value or a color in which a difference (R-G) value between the red component and a green component is greater than a threshold value. For example, when the color information is expressed by hue, saturation, and value brightness (HSC), the color likely to be a flame is a color in which an H value of a hue component is close to red and a V value of a brightness component is greater than a threshold value. The flame candidate region extracting unit 101A extracts a region including a pixel group indicating a color likely to be a flame out of the digitized pixels and sets the extracted region as the flame candidate region. In this case, the flame candidate region extracting unit 101A may exclude a region of which an area is less than a threshold value as noise out of the regions including the pixel groups showing the color likely to be a flame and set a region of which the area is equal to or greater than the threshold value as the first candidate region. The flame candidate region extracting unit 101A outputs information indicating the extracted flame candidate region to the flame determining unit 102A.

[0099] The flame determining unit 102A determines whether a flame is shown in the flame candidate region. The flame determining unit 102A determines whether a flame is shown in the flame candidate region on the basis of an index indicating shape complexity in the flame candidate region. The flame determining unit 102A calculates the index indicating shape complexity in the flame candidate region using a statistic of angles formed by a contour line connecting pixel groups constituting the contour of the flame candidate region.

[0100] For example, when the shape of the flame candidate region is a simple shape such as a circle, a triangle, a quadrangle, or a polygon, the angles formed by the contour line are concentrated on a specific angle. For example, when the shape of the flame candidate region is circular, the angles formed by the contour line are angles based on a curvature of the circle and a resolution of the image. When the shape of the flame candidate region is triangular, the angles formed by the contour line are one of angles formed by three vertices and an angle (180 degrees) formed by a straight line. When the shape of the flame candidate region is quadrangular, the angles formed by the contour line are one of angles (90 degrees in a square or a rectangle) formed by four vertices and an angle (180 degrees) formed by a straight line. For example, a fluorescent light is displayed on an image as a combination of substantially parallelograms, and this substantially parallelogram includes angles formed by vertices and the angle (180 degrees) formed by a straight line in many parts. The shape of a flame is not a simple shape described above, but is a complex shape in general. A contour line of a flame includes various angles and are not generally concentrated on a specific angle. By employing complexity of such a shape of a flame, the flame determining unit 102A according to this embodiment calculates an index indicating shape complexity using a statistic of angles formed by the contour line connecting the pixel groups constituting the contour of the flame candidate region. Then, the flame determining unit 102A determines whether a flame is shown in the flame candidate region on the basis of the calculated index. A specific method by which the flame determining unit 102A calculates the angles formed by the contour line will be described later in detail.

[0102] The determination result output unit 103 A outputs a determination result from the flame determining unit 102A, that is, whether a flame is shown in the target image. For example, the determination result output unit 103A may be configured to display the determination result by displaying the determination result on a display which is not illustrated. In this case, when a flame is shown in the target image, a user’s attention may be attracted by displaying a message indicating that there is a likelihood of a fire, outputting an alarm sound, or the like.

[0103] The image information storage unit 104A stores image information. Image information is image information of a target image. The image information may include information indicating the flame candidate region extracted by the flame candidate region extracting unit 101 A, the determination result from the flame determining unit 102A, or information indicating an index or the like used for the determination. For example, the image information storage unit 104A is constituted by a storage medium such as a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), or a read only memory (ROM) or an arbitrary combination of such storage media.

[0104] A specific method by which the flame determining unit 102A calculates the angles formed by the contour line will be described below with reference to FIG. 14 (FIGS. 14A to 14C). FIG. 14 is a diagram illustrating a process that is performed by the flame determining unit 102A according to this embodiment. In FIG. 14, target images TG and pixels corresponding to a flame candidate region R extracted in each target image TG are schematically illustrated. In FIG. 14, pixels Pl to Pl 7 are illustrated as a pixel group constituting the contour of the flame candidate region R.

[0105] As illustrated in FIG. 14A, the flame determining unit 102A selects one pixel, that is, a pixel Pll in this example, out of the pixels Pl to P17 which is the pixel group constituting the contour of the flame candidate region RA, as a start point B. Then, the flame determining unit 102A selects a pixel located on one side from the start point B and a pixel located on another side from the pixel group constituting the contour of the flame candidate region R. In the example illustrated in this drawing, the pixel P14 located on the left-upper side from the start point B is selected as a point A located on one side, and the pixel P8 located on the right-upper side from the start point B is selected as a point C located on another side.

[0106] As illustrated in FIG. 14B, the flame determining unit 102A selects three pixels from a combination different from the combination from which the pixels are selected in FIG. 14A. In the example illustrated in this drawing, the flame determining unit 102A selects the pixel P16 as a start point E. In the example, the flame determining unit 102A selects the pixel Pl3 located on the lower side from the start point E as a point D located on one side and selects the pixel P3 located on the right side from the start point E as a point F located on another side.

[0107] As illustrated in FIG. 14C, three pixels are selected from a combination different from the combination from which the pixels are selected in FIGS. 14A and 14B. Tn this example illustrated in this drawing, the flame determining unit 102A selects the pixel P7 as a start point H. In the example, the flame determining unit 102A selects the pixel P4 located on the left-upper side from the start point H as a point G located on one side and selects the pixel PIO located on the left-upper side from the start point H as a point K located on another side.

[0108] In this way, the flame determining unit 102A selects three pixels from the pixel group constituting the contour of the flame candidate region RA. In this case, the flame determining unit 102A may select three pixels which are not adjacent from the pixel group constituting the contour of the flame candidate region RA. In general, since pixels are arranged in a lattice shape, an angle formed by line segments connecting three adjacent pixels is a specific angle such as 0 degrees, 45 degrees, 90 degrees, 135 degrees, or 180 degrees, and the angle formed by the contour line are likely to be concentrated on the specific angle even when the shape of the flame candidate region is complex. For example, the flame determining unit 102A selects N (where N is an integer equal to or greater than 2) from the pixel group constituting the contour of the flame candidate region RA. Accordingly, it is possible to quantitatively detect features in which the angles formed by the contour line include various angles when the shape of the flame candidate region is complex and the angles formed by the contour line are concentrated on a specific angle when the shape of the flame candidate region is a simple circle or rectangle.

[0110] Specifically, the flame determining unit 102A extracts a numerical progression PAn indicating coordinates of the pixel group constituting the contour as contour information. For example, numerical progression PAn is a numerical progression in which pixel groups constituting the contour are sequentially arranged in a predetermined direction, for example, in a clockwise direction or a counterclockwise direction. The flame determining unit 102A selects three pixels which are separated by a predetermined distance from the numerical progression PAn. Specifically, the flame determining unit 102A selects three pixels (PAk-dA, PAk, and PAk+dA) separated by a distance dA from the numerical progression PAn {PAI, PA2, ..., PAn}.

[0111] The flame determining unit 102A calculates an angle formed by line segments which are formed by connecting the selected three pixels along the contour. Specifically, the flame determining unit 102A calculates vectors VAI and VA2 which are directed from a start point PAk to other points (PAk-dA) and (PAk+dA) in the coordinates of the three pixels (PAk-dA, PAk, and PAk+dA) selected from the numerical progression PAn. The vector VAI is a vector directed from the pixel (PAk) to the pixel (PAk-dA). The vector VA2 is a vector directed from the pixel (PAk) to the pixel (PAk+dA). For example, the flame determining unit 102A calculates an angle 0A formed by line segments using an inner product as represented by Expression (2).

[0112] dAx! *dAx2+dAy1 *dAy2 =^(dAxlA2+dAylA2M(dAx2A2+dAy2A2)xcos0A .. .(2) Here, (dAx 1, dAyl): coordinate values of vector VAI (dAx2, dAy2): coordinate values of vector VA2 0A: angle formed by line segments

[0113] In this way, the flame determining unit 102A calculates an angle formed by line segments connecting three pixels selected from the pixel group constituting the contour of the flame candidate region RA along the contour as an angle formed by the contour line. Specifically, the flame determining unit 102A calculates an angle ABC formed by line segments ABC in FIG. 14A, an angle DFE formed by line segments DEF in FIG. 14B, and an angle GHK formed by line segments GHK in FIG. 14C as the angles formed by the contour line.

[0114] For example, the flame determining unit 102A calculates angles formed by line segments with each of all the pixels constituting the contour as a start point as the angles formed by the contour line. The flame determining unit 102A calculates an index indicating shape complexity using a statistic of the angles formed by the contour line.

[0115] For example, the static may be arbitrarily determined as long as it is a statistic serving as an index indicating whether the angles formed by the contour line are concentrated on a specific angle such as 180 degrees. For example, an average value, a representative value, a mode, a maximum value, a minimum value, a variance, a standard deviation, and a combination thereof can be used as the statistic.

[0116] A specific method by which the flame determining unit 102A calculates an index using the statistic will be described below. In this embodiment, two indices including (1) a degree of concentration and (2) a deviation of angles which are modes are used as the index indicating shape complexity of a flame candidate region.

[0117] (1) The degree of concentration is an index indicating a degree to which the angles formed by the contour line are concentrated on a specific angle and is a ratio of the number of angles which are modes to the total number of angles formed by the contour lines. For example, when line segments with each of all the pixels constituting the contour as a start point are generated and angles formed by the generated line segments are acquired, the total number of angles formed by the contour line is the same as the total number of pixels constituting the contour line.

[0118] For example, when the total number of angles formed by the contour line is 100, an angle which is a mode is 180 degrees, and the number of angles of 180 degrees is 67, the degree of concentration is 0.67 (=67 / 100). When the total number of angles formed by the contour line is 100, an angle which is a mode is 7 degrees, and the number of angles of 7 degrees is 6, the degree of concentration is 0.06 (=6 / 100).

[0119] (2) The deviation of angles which are modes is an index indicating a degree to which the modes of the angles formed by the contour line in the images are not concentrated, that is, a degree of deviation, when a moving image obtained by capturing the monitoring target T is used. For example, the flame determining unit 102A calculates a mode in each target image constituting the moving image. Here, the mode is an angle which is the mode out of the angles formed by the contour line in the flame candidate region extracted from the target image.

[0120] For example, when the total number of target images constituting the moving image is 100 and the angles which are modes in the images includes various values 17 degrees, 3 degrees, 69 degrees, 41 degrees, ..., the deviation increases. On the other hand, when the angles which are modes in the images are concentrated on 180 degrees or the vicinity of 180 degrees such as 180 degrees, 181 degrees, 179 degrees, ..., the deviation decreases. A statistic such as a variance or a standard deviation may be used as the degree of deviation.

[0121] FIG. 15 is a flowchart illustrating a process flow that is performed by the flame detection device 10A according to this embodiment. The flame detection device 10A acquires a target image (Step S10A). For example, the flame detection device 10A acquires an image in which a monitoring target TA is imaged by the camera CAA as the target image. Then, the flame detection device 10A extracts a flame candidate region from the target image (Step S11A). The flame detection device 10A extracts a region with a color like a flame as the flame candidate region on the basis of color information in the target image. Then, the flame detection device 10A determines whether a flame is shown in the flame candidate region extracted in Step SHA (Step S12A). The flame detection device 10A determines whether a flame is shown in the flame candidate region on the basis of an index indicating shape complexity of the flame candidate region. Then, the flame detection device 10A outputs a determination result of Step S12A (Step S13A), The flame detection device 10A displays the determination result to be visible or outputs an alarm sound corresponding to the determination result.

[0122] FIG. 16 is a flowchart illustrating Step SI 1A of FIG. 15, that is, a process flow of extracting a flame candidate region from a target image. The flame detection device 10A digitalizes all the pixels in the target image according to whether each pixel value indicates a color likely to be a flame (Step SHOA). For example, when the color information is expressed in RGB, the flame detection device 10A determines that a pixel in which the R value is greater than a threshold value or a pixel in which the (R-G) value is greater than a threshold value is a pixel of a color like a flame. Alternatively, when the color information is expressed in HSV, the flame detection device 10A determines that a pixel of which the H value is close to red and the V value is greater than a threshold value is a pixel of a color like a flame. The flame detection device 10A extracts a region constituted by a pixel group indicating a color like a flame out of the digitized pixels (Step S111 A). The flame detection device 10A selects a region with an area equal to or greater than a threshold value out of the extracted regions as a flame candidate region (Step S112A).

[0123] FIG. 17 is a flowchart illustrating Step S12A of FIG. 15, that is, a process flow of determining whether a flame is shown in a flame candidate region. The flame detection device 10A acquires a pixel group constituting a contour of the flame candidate region extracted in S11A (Step S120A). The flame detection device 10A selects three pixels from the acquired pixel group constituting the contour (Step S121A). The flame detection device 10A calculates an angle formed by line segments connecting the three pixels along the contour (Step S122A).

[0124] The flame detection device 10A calculates an angle formed by line segments with each of the pixels constituting the contour as a start point (Step SI 23A). The flame detection device 10A prepares, for example, a histogram representing a distribution of angles as a statistic of the calculated angles (Step S I 24A). For example, the flame detection device 10A prepares a histogram in which the number of angles is correlated with each angle every 5 degrees. The flame detection device 10A calculates a degree of concentration as an index indicating shape complexity on the basis of the prepared histogram (Step S125A).

[0125] The flame detection device 10A determines whether a degree of concentration has been calculated for all the target images (Step SI26A). When a target image in which the degree of concentration has not been calculated remains, the flame detection device 10A causes the process flow to return to Step S10A.

[0126] On the other hand, when a target image in which the degree of concentration has not been calculated does not remain, the flame detection device 10A determines whether the degree of concentration is less than a threshold value (Step S127A). For example, the flame detection device 10A determines whether the degree of concentration calculated from each of all the target images in which the index has been calculated is less than the threshold value. In this case, for example, when the degree of concentration is less than the threshold value, the flame detection device 10A determines that a flame is shown in the flame candidate region in the corresponding target image. Alternatively, the flame detection device 10A may determine whether an average value of the degrees of concentration calculated from all the target images in which the index has been calculated is less than a threshold value. When the degree of concentration (or the average value of the degrees of concentration) is less than the threshold value, the flame determining unit 102A determines that a flame is shown in the flame candidate region (Step SI28A). On the other hand, when the degree of concentration (or the average value of the degrees of concentration) is equal to or greater than the threshold value, the flame determining unit 102Adetermines that a flame is not shown in the flame candidate region (Step SI29A).

[0127] In the example illustrated in the flowchart, the degree of concentration is used as the index, but the present disclosure is not limited thereto. The degree of deviation of the modes calculated from a plurality of target images constituting a moving image may be used as the index. For example, when a variance of the modes calculated from the target images is equal to or greater than a threshold value, the deviation of the modes of the target images is large, and thus the flame detection device 10A determines that a flame is shown in the flame candidate region. On the other hand, when the variance of the modes calculated from the target images is less than the threshold value, the deviation of the modes of the target images is small, and thus the flame detection device 10A determines that a flame is not shown in the flame candidate region.

[0128] An example of a target image, a flame candidate region, a statistic, and an index according to this embodiment will be described below with reference to FIGS. 18 to 21. FIGS. 18 to 21 are diagrams illustrating an example of a target image, a flame candidate region, a statistic, and an index according to this embodiment. In FIGS. 18 to 21, a target image TG, a flame candidate region 1 IK extracted from the target image TG, a histogram HG of angles formed by the contour of the flame candidate region HK, and an index SH (a degree of concentration) calculated from a statistic of the angles formed by the contour are illustrated.

[0129] FIG. 18 illustrates an example of a target image TG in which a flame is imaged. The flame candidate region HK extracted from the target image TG has a complex shape. In this case, the histogram HG of the angles formed by the contour of the flame candidate region HK represents that the flame candidate region HK is constituted by various angles of 0 degrees to 180 degrees. The degree of concentration is 0.082, and it can be seen from the index that the flame candidate region HK does not have a shape concentrated on a specific angle but has a complex shape.

[0130] FIG. 19 illustrates an example in which a shielding object is superimposed on the target image TG illustrated in FIG. 18. When the target image TG is shielded in this way, the flame candidate region HK extracted from the target image TG has a shape not including the shielded part. In this case, the histogram HG of the angles formed by the contour of the flame candidate region HK represents that the flame candidate region HK is constituted by various angles of 0 degrees to 180 degrees. The degree of concentration is 0.114, and it can be seen from the index that it can be determined as a flame even when a part of the flame candidate region HK is shielded.

[0131] FIG. 20 illustrates an example of a target image TG in which a circular object is imaged. A simple circular shape is extracted from the target image TG. In this case, the histogram HG of the angles formed by the contour of the flame candidate region HK represents that there are many specific angles and that the flame candidate region HK is constituted by angles including the specific angle. The degree of concentration is 0.516, and it can be seen from the index that the flame candidate region HK has a simple shape concentrated on the specific angle.

[0132] FIG. 21 illustrates an example of a target image TG in which a rectangular object is imaged. A simple rectangular shape is extracted from the target image TG. In this case, the histogram HG of the angles formed by the contour of the flame candidate region HK represents that there are many specific angles and that the angles are concentrated on a specific angle and the flame candidate region HK is constituted by angles including the specific angle. The degree of concentration is 0.709, and it can be seen from the index that the flame candidate region HK has a simple shape concentrated on the specific angle.

[0133] An example of an index when a target image is a moving image will be described below with reference to FIG. 22. In the example illustrated in FIG. 22, the index in the left part is calculated from a moving image in which a flame is imaged, and the index in the right part is calculated from a moving image in which a vehicle is imaged. In the upper part of FIG. 22, an example in which a degree of concentration is used as the index is illustrated, where the horizontal axis represents a frame image number which is a time axis and the vertical axis represents the degree of concentration. In the lower part of FIG. 22, an example in which a mode (a supplementary angle which is the mode) is used as the index is illustrated, where the horizontal axis represents a frame image number which is a time axis and the vertical axis represents the mode. The supplementary angle is (180 - angle).

[0134] As illustrated in the upper part of FIG. 22, when a moving image in which a flame is imaged is used, the degree of concentration in any image is less than 0.2. On the other hand, when a moving image in which a vehicle is imaged is used, the degree of concentration in any image is equal to or greater than 0.2. In this way, even when a moving image which is successively captured in a time series is used, it is possible to accurately determine whether an object is a flame using the degree of concentration.

[0135] In the lower part of FIG. 22, a temporal deviation of angles is illustrated using a supplementary angle (= 180 - angle). Since “supplementary angle = 180 - angle,” a deviation of the supplementary angles means a deviation of the angles. As illustrated in the lower part of FIG. 22, when a moving image in which a flame is imaged is used, the mode (the supplementary angle which is the mode) changes variously by images, that is, the deviation is large. On the other hand, when a moving image in which a vehicle is imaged is used, the mode in any image is 0 (zero), and there is no deviation. In this way, even when a moving image which is successively captured in a time series is used, it is possible to accurately determine whether an object is a flame using the degree of deviation of the modes.

[0136] As described above, the flame detection device 10A according to this embodiment includes the image acquiring unit 100A, the flame candidate region extracting unit 101A, and the flame determining unit 102A. The image acquiring unit 100A acquires a target image. The flame candidate region extracting unit 101A extracts a flame candidate region from the target image. The flame determining unit 102A determines whether a flame is shown in the flame candidate region. The flame determining unit 102A determines whether a flame is shown in the flame candidate region on the basis of an index indicating shape complexity of the flame candidate region. The flame determining unit 102A calculates the index using a statistic of the angles formed by the contour line connecting the pixel groups constituting the contour of the flame candidate region.

[0137] Accordingly, the flame detection device 10A according to this embodiment determines whether a flame is shown in the flame candidate region on the basis of the index indicating shape complexity. Since the flame detection device 10A calculates the index using a statistic of the angles formed by the contour line of the flame candidate region, it is possible to prevent a region having a shape such as a simple rectangular shape from being erroneously determined as a flame. Accordingly, it is possible to accurately detect a flame using a method different from the determination using circularity.

[0138] In the flame detection device 10A according to this embodiment, the flame determining unit 102A calculates an angle formed by line segments connecting three pixels selected from the pixel group constituting the contour of the flame candidate region along the contour as the angle formed by the contour line. The flame determining unit 102A calculates the index using a statistic of the angles formed by a plurality of line segments identified for each combination of pixels to be selected. Accordingly, the flame detection device 10A according to this embodiment can calculate the angle formed by the contour line as the angle formed by the line segments connecting the three pixels selected from the pixel groups constituting the contour of the flame candidate region along the contour, easily calculate the statistic of the angles formed by the contour line, and easily determine whether a flame is shown in the flame candidate region.

[0139] Tn the flame detection device 10A according to this embodiment, the flame determining unit 102A calculates an angle formed by line segments connecting three pixels which are not adjacent and which are selected from the pixel groups constituting the contour of the flame candidate region along the contour as the angle formed by the contour line. Accordingly, the flame detection device 10A according to this embodiment can avoid a combination of three pixels which are adjacent and avoid a situation in which the angles formed by the contour line are apparently concentrated on a specific angle and erroneous determination is performed even if the shape is complex.

[0140] In the flame detection device 10A according to this embodiment, the flame determining unit 102A calculates an angle formed by connecting three pixels which are separated by N pixels (where N is an integer equal to or greater than 2) out of the pixel group constituting the contour line along the contour as the angle formed by the contour line. Accordingly, the flame detection device 10A according to this embodiment can calculate the angle formed by line segments generated in a combination of the three pixels separated by almost the same distance and calculate an index more accurately indicating shape complexity in comparison with a case in which the distance between pixels is various such as being small or large.

[0141] In the flame detection device 10A according to this embodiment, the flame determining unit 102A calculates the index using the degree of concentration, that is, a ratio of the number of angles which are modes to the total number of angles formed by the contour line, as a statistic. Accordingly, with the flame detection device 10A according to this embodiment, it is possible to quantitatively represent the shape complexity as a degree of concentration of the angles of a shape on a specific angle.

[0142] In the flame detection device 10A according to this embodiment, the image acquiring unit 100A acquires a moving image, that is, a plurality of target images in which a monitoring target is captured in a time series. The flame determining unit 102A calculates a deviation of angles which are modes out of the angles formed by the contour line corresponding to each of the plurality of target images as an index. Accordingly, with the flame detection device 10A according to this embodiment, it is possible to determine that an object of which the shape is complex and changes from time to time is a flame and to perform more accurate determination.

[0143] In the flame detection device 10A according to this embodiment, the flame candidate region extracting unit 101A extracts a flame candidate region on the basis of color information in an image. Accordingly, with the flame detection device 10A according to this embodiment, it is possible to extract the flame candidate region on the basis of a color and to more easily extract the flame candidate region.

[0144] A modified example of the flame detection system according to this embodiment will be described below. This modified example is different from the aforementioned embodiment, in that a target image is enlarged or reduced (resized). Depending on a distance between the camera CAA capturing a target image and a region in which a flame breaks out, a contour of a flame may be deformed in image and become unclear or the shape of the flame may be much larger than the size of the image. In this modified example, in this case, a decrease in accuracy for extracting a flame candidate region is curbed by enlarging or reducing the target image.

[0146] For example, when the camera CAA is installed, a detection frame with a fixed size (for example, a lateral width of 10 m and a height of 5 m) is set for the target image. For example, a frame indicating a region close to the camera CAA such as a detection frame in which pixel coordinates (x, y) corresponding to four corners are (30, 100), (530, 100), (530, 350), and (30, 350) is set as a first detection frame. A frame indicating a region far from the camera CAA such as a detection frame in which pixel coordinates (x, y) corresponding to four corners are (130, 80), (430, 80), (430, 230), and (130, 230) is set as a second detection frame. A direction from top to bottom in an image is defined as a positive direction in the y axis. An axis perpendicular to the y axis is defined as the axis.

[0147] When such detection frames are set, for example, the size of the first detection frame in the image is 500x250 pixels, and the size of the second detection frame is 300x150 pixels. When a target image in which these detection frames are set is acquired, for example, the flame detection device 10A extracts the detection regions and enlarges or reduces the extracted detection regions to a uniform size such as 200x125 pixels. Accordingly, the size of an actual space which is imaged in one pixel is set to the same size regardless of the distance from the camera CAA. Accordingly, when a flame with the same size occurs, a detection result can be prevented from differing depending on whether a position is close to or far from the camera CAA.

[0148] When an image in the detection frame is enlarged or reduced, the contour can be unclear at a much larger enlargement / reduction ratio. Accordingly, it is preferable to appropriately set a combination of the size of each detection frame and the enlargement / reduction ratio.

[0149] The flame detection device 10A performs the resizing process before the flame candidate region is extracted or before it is determined whether a flame is shown in the flame candidate region. For example, by performing the resizing process before the flame candidate region is extracted, the flame detection device 10A can accurately extract the flame candidate region. By performing the resizing process before it is determined whether a flame is shown in the flame candidate region, it is possible to accurately determine whether a flame is shown in the flame candidate region.

[0150] As described above, in the flame detection device 10A according to the modified example of this embodiment, the image acquiring unit 100A acquires a target image in which a plurality of detection frames set according to a combination of a distance from the camera CAA and the size of the region in an actual space are provided. The flame candidate region extracting unit 101A extracts an image region in the detection frame from the target image and enlarges or reduces the extracted image region at a multiple which is set in correlation with the detection frame. The flame candidate region extracting unit 101A extracts the flame candidate region from the image using the image enlarged or reduced at the multiple. When the flame candidate region to be determined is a region in the detection frame, the flame determining unit 102A enlarges or reduces the flame candidate region at the multiple set in correlation with the detection frame. The flame determining unit 102A determines whether a flame is shown in the flame candidate region using the flame candidate region enlarged or reduced at the multiple.

[0151] (Third embodiment) A third embodiment of the present disclosure will be described below with reference to the accompanying drawings.

[0152] (Configuration of flame detection system IB) FIG. 23 is a diagram illustrating an example of a configuration of a flame detection system IB to which an image generation device 10B according to this embodiment is applied. For example, the flame detection system IB includes a camera CAB, an image generation device 10B, an operation control device 20B, and a smoke detection device 30B.

[0153] The camera CAB, the image generation device 10B, and the smoke detection device 30B are communicatively connected by a switch SW1. The operation control device 20B and the smoke detection device 30B are communicatively connected by a switch SW2. For example, communication among the camera CAB, the image generation device 10B, the operation control device 20B, and the smoke detection device 30B is realized by short-range communication using a communication network or a wireless local area network (LAN) or via a universal serial bus (USB) cable or the like. The camera CAB and the smoke detection device 30B may be provided as a unified body. That is, the camera CAB may be incorporated into the smoke detection device 30B.

[0154] The flame detection system 1B includes a fire detection mode and an operation confirmation mode. The fire detection mode is a mode in which a fire is detected on the basis of an image and is realized using the camera CAB and the smoke detection device 30B. Tn this case, the camera CAB and the smoke detection device 30B are connected by the switch SW1. The operation confirmation mode is a mode in which it is confirmed whether the smoke detection device 30B is operating correctly and is realized using the camera CAB, the image generation device 10B, the operation control device 20B, and the smoke detection device 30B. Tn this case, the camera CAB and the image generation device 10B are connected by the switch SW1, and the smoke detection device 30B and the operation control device 20B are connected by the switch SW2.

[0155] A target region TB is a region or an object which is monitored by the flame detection system IB. The target region TB may be arbitrarily set. For example, the target region TB is a roadway, tunnel premises, a building, or building premises.

[0156] The camera CAB images the target region TB. The camera CAB may capture a moving image or may capture a still image. The camera CAB outputs a captured image to the image generation device 10B or the smoke detection device 3 0B under the control of the switch SW1 by the operation control device 20B. Specifically, in the fire detection mode, the camera CAB outputs the captured image to the smoke detection device 30B. On the other hand, in the operation confirmation mode, the camera CAB outputs the captured image to the image generation device 10B.

[0157] For example, the image generation device 1 OB is a computer and is realized by a personal computer (PC), a server device, or the like. The image generation device 10B acquires a target image TGB in which the target region TB is imaged from the camera CAB and generates a synthetic image GG in which a smoke image (an image including smoke) is synthesized into the acquired image in the operation confirmation mode. The image generation device 10B outputs the generated synthetic image GG to the operation control device 20B. The synthetic image GG generated by the image generation device 10B is used to determine whether the smoke detection device 30B is operating correctly. A method by which the image generation device 1 OB generates the synthetic image GG will be described later in detail.

[0158] For example, the operation control device 20B is a computer and is realized by a personal computer (PC), a server device, or the like. The operation control device 20B causes the flame detection system IB to operate in the fire detection mode or the operation confirmation mode by controlling the switches SW1 and SW2.

[0159] In the operation confirmation mode, the operation control device 20B outputs the synthetic image GG generated by the image generation device 10B to the smoke detection device 30B. Then, the operation control device 20B acquires a determination result KK indicating whether a fire has broken out from the smoke detection device 30B. The operation control device 20B performs operation confirmation of determining whether the smoke detection device 30B is operating correctly on the basis of the synthetic image GG and the determination result KK.

[0160] For example, the smoke detection device 30B is a computer and is realized by a personal computer (PC), a server device, or the like. The smoke detection device 30B is provided in the target region TB. In the fire detection mode, the smoke detection device 30B acquires a target image TGB from the camera CAB and determines whether smoke is shown in the acquired image. In the operation confirmation mode, the smoke detection device 30B acquires the synthetic image GG from the operation control device 20B and determines whether smoke is shown in the acquired image. An arbitrary method may be employed as a method by which the smoke detection device 30B detects smoke on the basis of an image, and detailed description thereof will be omitted in this embodiment.

[0161] (Configuration of image generation device 10B) FIG. 24 is a block diagram illustrating an example of a configuration of the image generation device 10B according to this embodiment. For example, the image generation device 10B includes an image acquiring unit 100B, a generation unit 101B, an input unit 102B, a display unit 103B, an output unit 104B, and an image information storage unit 105B. The functional units (the image acquiring unit 100B and the generation unit 101B) included in the image generation device 10B are realized by causing a central processing unit (CPU) provided as hardware in the image generation device 10B to execute a program.

[0162] The image acquiring unit 100B acquires image information of the target image TGB captured by the camera CAB. The image acquiring unit 100B outputs the acquired image information to the generation unit 10IB.

[0163] For example, the input unit 102B includes an input device such as a mouse, a keyboard, a touch panel, a track ball, or a game controller. The input unit 102B acquires information which is input through a user’s operation input. The input unit 102B outputs the acquired information to the generation unit 101B.

[0164] For example, the display unit 103B includes a display device such as a liquid crystal display and displays an image under the control of a display control unit (not illustrated). For example, the display unit 103B displays the target image TGB and the synthetic image GG. The display unit 103B displays a tool for supporting an operation of allowing a user to generate the synthetic image GG such as a guide GD which will be described later (see FIG. 26).

[0165] The output unit 104B outputs image information of the synthetic image GG generated by the generation unit 101B to the operation control device 20B. When the image generation device 10B and the operation control device 20B are communicatively connected via a cable or the like, the output unit 104B is an output port of the cable. When the image generation device 10B and the operation control device 20B are communicatively connected via a communication network or the like, the output unit 104B is a communication module that communicates with the operation control device 20B via the communication network.

[0166] The image information storage unit 105B stores image information. The image information is image information of a target image. The image information includes image information of the target image TGB, the smoke image, and the synthetic image GG. The image information may include parameters which are applied when the synthetic image GG is generated such as information indicating a coefficient AB which will be described later. For example, the image information storage unit 105B is constituted by a storage medium such as a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), or a read only memory (ROM) or an arbitrary combination of such storage media.

[0167] The generation unit 101B generates a synthetic image GG in which a smoke image is synthesized into the target image TGB. Accordingly, the generation unit 101B can generate a synthetic image GG in which a fire seems to break out in the site from the target image TGB captured with the site in which the smoke detection device 30B is installed as the target region TB. A method of generating the synthetic image will be described below in detail.

[0168] The generation unit 101B generates the synthetic image GG by synthesizing a smoke image into the target image TGB. The generation unit 10IB calculates pixel values of the synthetic image GG using Expression (3).

[0169] IGG=lBG+ABxlsmoke ...(3) Here, IBG: pixel value of target image TGB AB: coefficient IGG: pixel value of synthetic image GG

[0170] A coefficient AB in Expression (3) may be arbitrarily set. For example, the coefficient AB is a real number from -1 to +1. A color of smoke can be set by a sign of the coefficient AB. For example, when the coefficient AB is set to a positive value, white smoke (white smoke) is synthesized. When the coefficient AB is set to a negative value, black smoke (black smoke) is synthesized. A color depth of smoke can be set using the absolute value of the coefficient AB. For example, when the coefficient AB is set to a value close to 0 (zero), light smoke is synthesized. When the absolute value of the coefficient AB is set to a value close to a maximum value (for example, 1), dark smoke is synthesized.

[0171] Here, a blending will be described. The a blending is a technique of synthesizing two images using a coefficient (an a value). For example, the a blending is used to synthesize an image in which a character or the like is provided to be visible through the background. When a synthetic image in which a background image and an object image are synthesized is generated, pixel values of the synthetic image are calculated using Expression (4) in a blending. The background image is an image including the background. The object images is an image including an object such as a character.

[0172] IGG=(l-aB)xIGl+aBxlG2 -(4) Here, 1G1: pixel value of background image IG2: pixel value of object image aB: coefficient IGG: pixel value of synthetic image GG

[0173] For example, the coefficient aB in Expression (4) is a real number in a range from 0 (zero) to +1. When a light object image which is almost transparent is synthesized, the coefficient aB is set to a small value. When a dark object image which is clearly visible is synthesized, the coefficient aB is set to a large value. The pixel values of the background image are adjusted according to the set value of the coefficient aB.

[0174] As represented by Expression (4), the pixel values of the background image are adjusted according to the coefficient aB in a blending. An effect in which an object such as a character seems to be natural in the background can be expected.

[0175] On the other hand, the generation unit 10IB according to this embodiment does not perform adjustment based on the coefficient AB on the pixel values of the target image TGB which is the background image as represented by Expression (3). Unlike an object such as a character, smoke is normally semitransparent. Accordingly, by not multiplying the pixel values of the background image by (1 -AB) in this embodiment, a natural synthetic image GG without discomfort in which smoke seems to go up in the target region TB can be expected to generate.

[0176] (Application Example 1) In Application Example 1, the generation unit 101B may generate a synthetic image GG using the coefficient AB which is constant for the pixels.

[0177] (Application Example 2) In Application Example 2, the generation unit 101B may perform a clipping process. The clipping process is a process of fixing a value calculated using Expression (3) to an upper limit when the value calculated using Expression (3) exceeds the upper limit (for example, 255) of the pixel values. The generation unit 101B may perform the clipping process on a lower limit. That is, the generation unit 101B may fix the value calculated using Expression (3) to a lower limit when the value calculated using Expression (3) exceeds than the lower limit (for example, 0) of the pixel values.

[0178] For example, the generation unit 101B performs the clipping process on the pixel values of the synthetic image GG using Expression (5) or Expression (6).

[0179] IGG=IMAX (IGGC>IMAX) IGG=IGGS (IGGC<IMAX) . ..(5) Here, IMAX: upper limit of pixel values (for example, 255) IGGC: pre-clipping pixel value IGG: pixel value of synthetic image GG

[0180] IGG=IMTN (TGGCcTMAX) 1GG=1GGC (IGGC>IMIN) ...(6) Here, IMIN: lower limit of pixel values (for example, 0) 1GGC: pre-clipping pixel value IGG: pixel value of synthetic image GG

[0181] (Application Example 3) In Application Example 3, the generation unit 101B may adjust the pixel values of the background image according to darkness of a smoke image, that is, the absolute value of the coefficient AB. Accordingly, when dark smoke is synthesized or the like, smoke can be synthesized into the background such that the smoke seems to be natural.

[0182] For example, the generation unit 101B calculates the pixel values of the synthetic image GG using Expression (7).

[0183] IGG=BBxIBG+ABxIsmoke ...(7) Here, IBG: pixel value of target image TGB Ismoke: pixel value of smoke image AB, BB: coefficient | AB| >threshold value TH BB<(1-AB) IGG: pixel value of synthetic image GG

[0184] (Configuration of operation control device 20B) FIG. 25 is a block diagram illustrating an example of a configuration of the operation control device 20B according to this embodiment. For example, the operation control device 20B includes a synthetic image acquiring unit 200B, a fire determination result acquiring unit 20IB, an operation determining unit 202B, and an image information storage unit 203B. The functional units (the synthetic image acquiring unit 200B, the fire determination result acquiring unit 201B, and the operation determining unit 202B) included in the operation control device 20B are realized by causing a central processing unit (CPU) provided as hardware in the image generation device 10B to execute a program.

[0185] The synthetic image acquiring unit 200B acquires image information of the synthetic image GG generated by the image generation device 10B. The synthetic image acquiring unit 200B outputs the acquired image information to the operation determining unit 202B.

[0186] The fire determination result acquiring unit 201B acquires a determination result KK from the smoke detection device 30B. The fire determination result acquiring unit 201B outputs the acquired determination result KK to the operation determining unit 202B.

[0187] The operation determining unit 202B determines whether the smoke detection device 30B is operating correctly on the basis of the synthetic image GG and the determination result KK. For example, when the smoke detection device 30B determines that a fire has broken out on the basis of the synthetic image GG including smoke, the operation determining unit 202B determines that the smoke detection device 30B is operating correctly. On the other hand, when the synthetic image GG including smoke is output to the smoke detection device 30B and the smoke detection device 30B determines that a fire has not broken out, the operation determining unit 202B determines that the smoke detection device 30B does not operate correctly. The operation determining unit 202B may display a determination result of whether the smoke detection device 30B is operating correctly as an operation confirmation result on a display unit (not illustrated) of the operation control device 20B.

[0188] The image information storage unit 203B stores image information. The image information is image information of a target image. The image information includes image information of the synthetic image GG. The image information may include parameters which are applied when the synthetic image GG is generated such as information indicating the coefficient AB which will be described later. For example, the image information storage unit 105B is constituted by a storage medium such as an HDD, a flash memory, an EEPROM, a RAM, or a ROM or an arbitrary combination of such storage media.

[0189] (Target image TGB and synthetic image GG) The target image TGB and the synthetic image GG will be described below with reference to FIGS. 26 and 27. FIG. 26 is a diagram illustrating an example of the target image TGB. FIG. 27 is a diagram illustrating an example of the synthetic image GG.

[0190] As illustrated in FIG. 26, for example, when image information of a target image TGB is acquired from the camera CAB, the image generation device 10B displays the target image TGB on the display unit 103B using the acquired image information. In this case, the image generation device 10B may display a support tool along with the target image TGB. The support tool is a tool for supporting an operation of generating a synthetic image GG desired by a user. For example, the image generation device 10B displays a slide switch SW, a check box CB, and a guide GD as the support tool. The slide switch SW is a switch for setting darkness of smoke. An amount of sliding of the slide switch SW interlinks with the coefficient AB, and a value of the coefficient AB corresponding to the amount of sliding is set. The check box CB is a check box for setting a color of smoke, that is, whether black smoke is to be synthesized or white smoke is to be synthesized. An item checked in the check box CB interlinks with the sign of the coefficient AB, and it is determined whether to set the coefficient AB to a positive value or a negative value.

[0192] The guide GD is an indicator for guiding a color of smoke corresponding to an object which is a fire source. For example, when gasoline or the like is combusted, black smoke is often generated. When the temperature of an electric device or the like increases abnormally or the like, white smoke may be generated. However, an operator who has installed the smoke detection device 30B or a person in charge in the site in which the smoke detection device 30B is installed may not have knowledge that the color of smoke differs according to a fire source. Accordingly, by displaying this guide GD to a user who generates the synthetic image GG, support is provided such that a color of smoke corresponding to an object is set. Accordingly, it is possible to generate a natural synthetic image GG without discomfort in which smoke of a color considered to be observed when a fire breaks out actually is being generated.

[0193] For example, the user sees the target image TGB displayed on the display unit 103B and selects an object generating smoke from an object group imaged in the target image TGB. In the example illustrated in the drawing, a vehicle, a traffic signal, and the like are imaged in the target image TGB. For example, the user selects a vehicle as the object generating smoke by clicking the vehicle with a mouse or the like. Then, the user sets black smoke or white smoke by performing an operation of checking the check box CB. At this time, the user sees a message displayed on the guide GD and selects a color corresponding to the object. For example, when the vehicle is selected, the user selects black smoke. Then, the user sets darkness of smoke to be synthesized by performing an operation of causing the slide switch SW to slide. For example, the image generation device 10B combines smoke of darkness interlinking with the sliding operation into the target image TGB and displays the synthetic image.

[0194] By performing this operation, the synthetic image GG illustrated in FIG. 27 is generated and displayed on the display unit 103B.

[0195] (Process flow that is performed by image generation device 10B) A process flow that is performed by the image generation device 10B will be described below with reference to FIG. 28. FIG. 28 is a flowchart illustrating a process flow that is performed by the image generation device 10B according to this embodiment.

[0196] The image generation device 10B acquires image information of a target image TGB (Step SI0B). The image generation device 10B displays the acquired target image TGB along with a support tool such as the guide GD (Step SUB). The image generation device 10B sets the coefficient AB based on an amount of sliding of the slide switch SW (Step S12B). For example, the image generation device 10B calculates the pixel values of the synthetic image GG using the set coefficient AB and Expression (3) (Step SI 3B). When the calculated pixel values exceed the upper limit of the pixels, the image generation device 10B may correct the pixel values by performing the clipping process (Step S14B). The image generation device 10B displays the generated synthetic image GG (Step S15B).

[0197] As described above, the image generation device 10B according to this embodiment includes the image acquiring unit 100B and the generation unit 101B. The generation unit 101B acquires a target image TGB in which the target region TB at the time of non-fire is imaged. The generation unit 101B generates a synthetic image GG in which a smoke image is synthesized into the target image TGB. The synthetic image GG is used to determine whether the smoke detection device 30B is operating correctly.

[0198] Accordingly, the image generation device 10B according to this embodiment can perform operation confirmation using an image of the site in which the smoke detection device 30B is installed. A smoke image can be synthesized into the image in which the site in which the smoke detection device 30B is installed is imaged as the target region TB. It is possible to perform operation confinnation using the realistic synthetic image GG and to confirm whether the smoke detection device 30B is operating correctly with higher accuracy. Since it is not necessary to use a three-dimensional shape model or to fonn a virtual space and the synthetic image GG can be generated using the target image TGB which is imaged in the actual site, it is possible to generate an image in which a fire seems to break out actually in the site without using a high-specification device. In the image generation device 10B according to this embodiment, the generation unit 101B generates a synthetic image GG in which sum values of values obtained by multiplying the pixel values (IBG) of the target image TGB and the pixel values of the smoke image (Ismoke) by a predetermined multiple (the coefficient AB) are used as pixel values. Accordingly, with the image generation device 10B according to this embodiment, it is possible to generate a natural synthetic image GG without discomfort using normal characteristics in which smoke is semitransparent.

[0200] In the image generation device 10B according to this embodiment, the generation unit 101B sets the predetermined multiple (the coefficient AB) to a positive value when a smoke image is expressed as white smoke and sets the predetermined multiple (the coefficient AB) to a negative value when the smoke image is expressed as black smoke. Accordingly, with the image generation device 10B according to this embodiment, it is possible to control a color of smoke with simple settings for setting whether to set the sign of the coefficient AB to a positive value or a negative value.

[0201] For example, when a pixel value is expressed using RGB of 0 to 255, white is expressed by a pixel value (255, 255, 255) and black is expressed by a pixel value (0, 0, 0). On the basis of these characteristics, in this embodiment, for example, the pixel values of the synthetic image GG is increased to approach white by setting the coefficient AB to a positive value in Expression (3). The pixel values of the synthetic image GG is decreased to approach black by setting the coefficient AB to a negative value.

[0202] The image generation device 10B according to this embodiment further includes the input unit 102B and the display unit 103B. The display unit 103B displays the guide GD along with the target image TGB. The guide GD is an indicator for guiding a color of smoke corresponding to an object which is a fire source. An operation from a user is input to the input unit 102B. The generation unit 101B generates a synthetic image GG in which a smoke image of a color designated through the operation input to the input unit 102B is synthesized into the target image TGB. Accordingly, with the image generation device 10B according to this embodiment, support can be performed such that a natural synthetic image GG without discomfort is generated through an operation from a user.

[0203] The flame detection system IB according to this embodiment includes the image acquiring unit 100B, the generation unit 101B, and the operation determining unit 202B. The flame detection system 1B is an example of a “movement configuration system.” The operation determining unit is an example of a “determination unit.” The operation determining unit 202B determines whether the smoke detection device 3 0B which is provided in the target region TB and which determines whether smoke is being generated in the target region TB is operating correctly using the synthesized image GG. Accordingly, with the flame detection system IB according to this embodiment, it is possible to confirm whether the smoke detection device 30B is operating correctly in the site using an image in which the site is imaged and to improve reliability of the smoke detection device 30B.

[0204] (Modified Example 1 of flame detection system according to this embodiment) Modified Example 1 of the flame detection system according to this embodiment will be described below. This modified example is different from the aforementioned embodiment, in that a smoke image with a size corresponding to a depth of a space imaged in a target image TGB is synthesized.

[0205] In this modified example, a depth of a space imaged in the target image TGB is detected by installing an object serving as a mark such as a marker M (see FIG. 29) in the target region TB.

[0206] FIG. 29 is a diagram illustrating a marker M. For example, the marker M is an object having a prescribed length as illustrated in FIG. 29. For example, the marker M is an object having a shape like a pole in which a sphere is attached to both ends. Such a marker M is installed in an imaging direction, and a space in which the marker M is installed is imaged by the camera CAB. Alternatively, a test image in which the marker M is installed is captured in advance, a plurality of spaces OY (see FIG. 31 A) are formed in a depth direction on the basis of the captured test image, and parameters used to generate the synthetic image GG may be set in advance to synthesize a smoke image with a size corresponding to the formed spaces OY. The marker M is not limited to the example illustrated in FIG. 29. The marker M may have an arbitrary shape as long as at least a marker and the size of the marker in the target image TGB can be confirmed. For example, the marker M may be constituted by a two-dimensional code or the like, the position of the marker may be automatically detected using a program or the like, and the size thereof may also be detected.

[0207] A method of synthesizing a smoke image with a size corresponding to a depth according to this modified example will be described below with reference to FIG. 30 and FIG. 31 (FIGS. 31A and 3 IB). FIG. 30 is a diagram illustrating an example of a target image TGB according to Modified Example 1 of this embodiment. FIG. 31 is a diagram illustrating an example of a synthetic image GG according to Modified Example 1 of this embodiment.

[0208] As illustrated in FIG. 30, in this modified example, a target image TGB (or a test image) including one or more markers (a plurality of markers M1 to M5 in the drawing) is captured and displayed on the display unit 103B. As illustrated in FIG. 31 A, the image generation device 10B detects a depth of a space on the basis of positions of the markers M and forms a plurality of spaces OY (spaces OY1 to OY5) corresponding to the detected depth. The spaces OY seem to have different sizes in the target image TGB but are spaces having the same size in an actual space. As illustrated in FIG. 3IB, the image generation device 10B determines the size of a smoke image to be synthesized according to in what space OY an object selected by a user is present. For example, when a smoke image is synthesized into a vehicle in the space OY5 which is located on the near side (the camera CAB side) in the imaging direction and which seems to be relatively large, the image generation device 10B combines a relatively large smoke image. On the other hand, when a smoke image is synthesized into a vehicle in the space OY2 which is located on a far side in the imaging direction and which seems to be relatively small, the image generation device 10B combines a relatively small smoke image. Accordingly, when a fire with the same scale is assumed, it is possible to more realistically reproduce a situation in which the size of smoke imaged in the target image TGB differs according to a place at which a fire breaks out. As described above, in the image generation device 10B according to Modified Example 1 of this embodiment, depth information indicating a depth of a target region TB such as image information of the markers M indicating a length is included in the target image TGB. The generation unit 101B generates a synthetic image GG in which a smoke image with a size corresponding to the depth information is synthesized into the target image TGB. Accordingly, in Modified Example 1 of the flame detection system according to this embodiment, it is possible to identify a depth of a space imaged in the target image TGB and to resize the size of the smoke image synthesized into an object present in a space corresponding to the depth according to the identified depth. Accordingly, when it is intended to generate smoke at any position in the target region TB, it is possible to generate a synthetic image GG close to the size of smoke which is actually generated.

[0210] (Modified Example 2 of flame detection system according to this embodiment) Modified Example 2 of the flame detection system according to this embodiment will be described below. This modified example is different from the aforementioned embodiment, in that a synthetic image GG is used for a fire drill. In a fire drill, it is important to create a situation close to an actual fire and to allow a participant to experience what happens in an actual fire and how to evacuate. In a normal fire drill, a situation close to an actual fire is created by giving a pseudo-signal to a fire alarm box or the like to output a fire alarm or the like. In this modified example, a synthetic image GG is displayed in the fire drill. Accordingly, a participant can be made to see a situation in which a space near the participant is on fire. As a result, it is possible to cause a participant to realistically experience what happens in an actual fire.

[0211] Some or all of the flame detection devices 10 and 10A, the flame detection system IB, the image generation device 10B, and the operation control device 20B according to the aforementioned embodiments may be realized by a computer. In this case, the functions may be realized by recording a program for realizing the functions on a computer-readable recording medium and causing a computer system to read and execute the program recorded on the recording medium. The “computer system” mentioned herein is a computer system including an operating system (OS) or hardware such as peripherals. The “computer-readable recording medium” may be, for example, a portable medium such as a flexible disk, a magneto-optical disc, a ROM, or a CD-ROM or a storage device such as a hard disk incorporated into a computer system. The “computer-readable recording medium” may include a medium that dynamically holds a program for a short time like a communication circuit line when a program is transmitted via a network such as the Internet or a communication circuit line such as a telephone circuit line or a medium that holds a program for a predetermined time like a volatile memory in a computer system which serves as a server or a client in that case. The program may be a program for realizing some of the aforementioned functions, may also be a program for realizing the aforementioned functions in combination with another program stored in the computer system, or may be a program for realizing the aforementioned functions using a programmable logic device such as an FPGA.

[0212] While embodiments of the present disclosure have been described above with reference to the drawings, specific configurations are not limited to the embodiments, and designs in a range not departing from the gist of the present disclosure is also included. The constituent elements in the embodiments, the modified examples, and the application examples can be appropriately combined. INDUSTRIAL APPLICABILITY

[0213] It is possible to provide a flame detection device, a flame detection method, an image generation device, a movement confirmation system, an image generation method, and a program in a flame detection system and a flame detection system that can detect a flame with high accuracy. REFERENCE SIGNS LIST

[0214] 1, 1A... Flame detection system 10,10A... Flame detection device 100, 100A... Image acquiring unit (acquisition unit) 101, 101 A... Flame candidate region extracting unit (extraction unit) 102, 102A... Flame determining unit (determination unit) 1B... Flame detection system (movement confirmation system) 10B... Image generation device 100B... Image acquiring unit 10IB... Generation unit 102B... Input unit 103B... Display unit 20B... Operation control device 202B... Operation determining unit (determination unit) 30B... Smoke detection device

Claims

1. A flame detection device comprising:an acquisition unit configured to acquire a plurality of target images in which a monitoring target is captured in a time series;an extraction unit configured to extract a flame candidate region from each target image acquired by the acquisition unit; anda determination unit configured to calculate an index indicating that an object shown in the flame candidate region moves like a flame using a degree to which an amount of movement of a tracking point set in the flame candidate region fluctuates with respect to an average value of the amount of movement and to determine whether a flame is shown in the flame candidate region on the basis of the calculated index.[Claim 2J The flame detection device according to claim 1, wherein the determination unit calculates an optical flow of a pixel included in the flame candidate region as the amount of movement.

3. The flame detection device according to claim 2, wherein the determination unit calculates the index using at least one of a first number of crossings in which the amount of movement changes from a value less than the average value to a value greater than the average value and a second number of crossings in which the amount of movement changes from a value greater than the average value to a value less than the average value.

4. The flame detection device according to claim 3, wherein the determination unit calculates the amount of movement and the average value in a first direction in the target images and a second direction other than the first direction and determines whether a flame is shown in the flame candidate region on the basis of the index in each of the

5. The flame detection device according to any one of claims 1 to 4, wherein the extraction unit extracts a first candidate region on the basis of color information included in the plurality of target images,wherein the extraction unit determines whether the first candidate region in a first image and the first candidate region in a second image are based on the same object on the basis of the first candidate region in the first image extracted from the first image out of the plurality of target images and the first candidate region in the second image extracted from the second image which is captured after the first image out of the plurality of target images, and determines the first candidate region in the first image and / or the first candidate region in the second image as a second candidate region when the two first candidate regions are based on the same object, andwherein the extraction unit determines the flame candidate region from the second candidate region.

6. The flame detection device according to claim 5, wherein the extraction unit calculates a degree to which the second candidate region is likely to be a flame on the basis of frequency characteristics in which a color in the second candidate region changes and determines whether the second candidate region is set to be the flame candidate region on the basis of the calculated degree of being like a flame.

7. A flame detection method that is performed by a flame detection device which is a computer, the flame detection method comprising:causing an acquisition unit to acquire a plurality of target images in which a monitoring target is captured in a time series;causing an extraction unit to extract a flame candidate region from each target image acquired by the acquisition unit; andcausing a determination unit to calculate an index indicating that an objectshown in the flame candidate region moves like a flame using a degree to which an amount of movement of a tracking point set in the flame candidate region fluctuates with respect to an average value of the amount of movement and to determine whether a flame is shown in the flame candidate region on the basis of the calculated index.

8. A program causing a flame detection device which is a computer to perform:acquiring a plurality of target images in which a monitoring target is captured in a time series;extracting a flame candidate region from each target image; andcalculating an index indicating that an object shown in the flame candidate region moves like a flame using a degree to which an amount of movement of a tracking point set in the flame candidate region fluctuates with respect to an average value of the amount of movement and determining whether a flame is shown in the flame candidate region on the basis of the calculated index.

9. A flame detection device comprising:an acquisition unit configured to acquire a target image in which a monitoring target is imaged;an extraction unit configured to extract a flame candidate region from the target image acquired by the acquisition unit; anda determination unit configured to calculate an index indicating shape complexity in the flame candidate region using a statistic of angles formed by contour lines connecting pixel groups constituting a contour of the flame candidate region and to determine whether a flame is shown in the flame candidate region on the basis of the calculated index.

10. The flame detection device according to claim 9, wherein the determination unit calculates the index using an angle formed by line segments connecting three pixelsselected from a pixel group constituting the contour of the flame candidate region along the contour as an angle formed by the contour lines and using the statistic of angles formed by each of a plurality of the line segments identified for each combination of the selected pixels.

11. The flame detection device according to claim 10, wherein the determination unit uses an angle formed by line segments connecting three pixels which are not adjacent and which are selected from pixel groups constituting the contour of the flame candidate region along the contour as the angle formed by the contour line.

12. The flame detection device according to claim 11, wherein the determination unit calculates an angle formed by connecting three pixels which are separated by N pixels (where N is an integer equal to or greater than 2) out of the pixel groups constituting the contour along the contour as the angle formed by the contour lines.

13. The flame detection device according to claim 12, wherein the determination unit calculates a ratio of the number of angles which are modes to the total number of angles formed by the contour lines as the index.

14. The flame detection device according to claim 12, wherein the acquisition unit acquires a plurality of target images in which the monitoring target is captured in a time series, andwherein the determination unit calculates a deviation of angles which are modes out of the angles formed by the contour lines corresponding to each of the plurality of target images as the index.

15. The flame detection device according to any one of claims 9 to 14, wherein the extraction unit extracts the flame candidate region on the basis of color information in an image.

16. A flame detection method that is performed by a flame detection device which is a computer, the flame detection method comprising:causing an acquisition unit to acquire a target image in which a monitoring target is imaged;causing an extraction unit to extract a flame candidate region from the target image acquired by the acquisition unit; andcausing a determination unit to calculate an index indicating shape complexity in the flame candidate region using a statistic of angles formed by contour lines connecting pixel groups constituting a contour of the flame candidate region and to determine whether a flame is shown in the flame candidate region on the basis of the calculated index.

17. A program causing a flame detection device which is a computer to perform:acquiring a target image in which a monitoring target is imaged;extracting a flame candidate region from the acquired target image; andcalculating an index indicating shape complexity in the flame candidate region using a statistic of angles formed by contour lines connecting pixel groups constituting a contour of the flame candidate region and determining whether a flame is shown in the flame candidate region on the basis of the calculated index.

18. An image generation device comprising:an acquisition unit configured to acquire a target image in which a target region is imaged; anda generation unit configured to generate a synthetic image in which a smoke image is synthesized with the target image acquired by the acquisition unit,wherein the synthetic image is used to determine whether a smoke detectiondevice which is provided in the target region and which determines whether smoke is being generated in the target region on the basis of an image is operating correctly.

19. The image generation device according to claim 18, wherein the generation unit generates the synthetic image with a sum value of a pixel value in the target image and a value obtained by multiplying a pixel value in the smoke image by a predetermined multiple as a pixel value.

20. The image generation device according to claim 19, wherein the generation unit sets the predetermined multiple to a positive value when the smoke image is represented as white smoke and sets the predetermined multiple to a negative value when the smoke image is represented as black smoke.

21. The image generation device according to claim 18, further comprising: a display unit configured to display the target image and a guide for guiding a color of smoke based on an object which is a tire source; andan input unit configured to receive an input of an operation from a user, wherein the generation unit generates the synthetic image in which a smoke image of a color designated by the operation input to the input unit is synthesized with the target image.

22. The image generation device according to claim 18, wherein depth information indicating a depth of the target region is included in the target image, and wherein the generation unit generates the synthetic image in which a smoke image with a size corresponding to the depth information is synthesized with the target image.

23. A movement confirmation system comprising:an acquisition unit configured to acquire a target image in which a target region is imaged;a generation unit configured to generate a synthetic image in which a smoke image is synthesized with the target image acquired by the acquisition unit; anda determination unit configured to determine whether a smoke detection device which is provided in the target region and which determines whether smoke is being generated in the target region on the basis of an image is operating correctly using the synthetic image.

24. An image generation method that is performed by an image generation device which is a computer, the image generation method comprising:causing an acquisition unit to acquire a target image in which a target region is imaged;causing a generation unit to generate a synthetic image in which a smoke image is synthesized with the target image acquired by the acquisition unit; anddetermining whether a smoke detection device which is provided in the target region and which determines whether smoke is being generated in the target region on the basis of an image is operating correctly using the synthetic image.

25. A program causing an image generation device which is a computer to perform:acquiring a target image in which a target region is imaged;generating a synthetic image in which a smoke image is synthesized with the acquired target image; anddetermining whether a smoke detection device which is provided in the target region and which determines whether smoke is being generated in the target region on the basis of an image is operating correctly using the synthetic image.

Images