Flame detection device, flame detection method, and program

Abstract

To accurately determine whether or not a flame is shown in an image even when a fire occurs in a movable body.SOLUTION: A flame detection device comprises: an acquisition unit which acquires a plurality of object images obtained by imaging a monitoring object in time-series; an extraction unit which extracts a flame candidate region from each object image acquired by the acquisition unit; and a determination unit which calculates such an index that an object shown in the flame candidate region shows a flame-like movement by using a fluctuation degree of a movement amount of a tracking point set in the flame candidate region with respect to an average value of the movement amount, and determines whether or not a flame is shown in the flame candidate region on the basis of the calculated index.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a flame detection device, a flame detection method, and a program. [Background technology]

[0002] A technology for detecting the occurrence of a fire using images is known (see, for example, Patent Document 1). A moving object present in a plurality of images captured in a time series is extracted, and if the extracted moving object behaves like a flame, the moving object is determined to be a flame. For example, if the coordinates of the moving object's center of gravity change periodically or monotonically increase or decrease, the moving object is determined not to be a flame but rather a red light whose light emission state changes periodically, or a moving object such as a vehicle moving in one direction. By excluding moving objects that behave in a manner that does not resemble a flame from a flame, moving objects such as vehicles with their red lights on are not mistakenly detected as a flame, thereby improving detection accuracy. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6140599 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are cases where the flame moves. For example, if a fire breaks out on a moving vehicle, the flame moves as the vehicle moves. When an image of such a moving flame is captured, it is desirable to be able to correctly detect that a flame is being displayed in the image.

[0005] The present invention has been made in consideration of the above circumstances, and its purpose is to provide a flame detection device, a flame detection method, and a program that can accurately determine whether or not a flame is shown in an image even when a fire occurs in a moving object. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, one embodiment of the present invention includes an acquisition unit that acquires a plurality of target images in which a monitoring target is captured in time series, an extraction unit that extracts a flame candidate region from each of the target images acquired by the acquisition unit, and a calculation unit that calculates an index indicating a flame-like movement of an object shown in the flame candidate region using the degree of fluctuation of the amount of movement of a tracking point set in the flame candidate region relative to the average value of the amount of movement. ,before The amount of movement of the tracking points set in the recording candidate area is the average of the aforementioned amounts of movement. or a second crossing frequency that changes from a value above the average value to a value below the average value. calculated , the object shown in the flame candidate region exhibits flame-like movement. The flame detection device includes a determination unit that determines whether or not a flame is indicated in the flame candidate region based on an index. Moreover, one embodiment of the present invention is a flame detection device comprising: an acquisition unit that acquires a plurality of target images of a monitored object captured in chronological order; an extraction unit that extracts a flame candidate region from each of the target images acquired by the acquisition unit; and a determination unit that calculates an index indicating that the object shown in the flame candidate region is moving in a flame-like manner as the degree to which the amount of movement of a tracking point set in the flame candidate region fluctuates relative to the average value of the amount of movement, for each of a first direction in the target image and a second direction different from the first direction, and determines whether a flame is shown in the flame candidate region based on the index in each direction.

[0007] Furthermore, one embodiment of the present invention is a flame detection method performed by a flame detection device that is a computer, in which an acquisition unit acquires a plurality of target images in which a monitoring target is photographed in time series, an extraction unit extracts a flame candidate region from each of the target images acquired by the acquisition unit, and a determination unit ,before The amount of movement of the tracking points set in the recording candidate area is the average of the aforementioned amounts of movement. or a second crossing frequency that changes from a value above the average value to a value below the average value. calculated , the object shown in the flame candidate region exhibits flame-like movement. The flame detection method determines whether or not a flame is shown in the flame candidate region based on an index. Moreover, one embodiment of the present invention is a flame detection method performed by a flame detection device that is a computer, in which an acquisition unit acquires a plurality of target images in which a monitored object is imaged in chronological order, an extraction unit extracts a flame candidate region from each of the target images acquired by the acquisition unit, and a determination unit calculates an index indicating that the object shown in the flame candidate region is moving in a flame-like manner as the degree to which the amount of movement of a tracking point set in the flame candidate region fluctuates relative to the average value of the amount of movement, for each of a first direction in the target image and a second direction different from the first direction, and determines whether a flame is shown in the flame candidate region based on the index in each direction.

[0008] In addition, one embodiment of the present invention is a method for causing a flame detection device that is a computer to acquire a plurality of target images in which a monitoring target is photographed in time series, and to extract a flame candidate region from each of the target images. ,before The amount of movement of the tracking points set in the recording candidate area is the average of the aforementioned amounts of movement. or a second crossing frequency that changes from a value above the average value to a value below the average value. calculated , the object shown in the flame candidate region exhibits flame-like movement. The program determines whether or not a flame is shown in the flame candidate region based on the index. Furthermore, one embodiment of the present invention is a program that causes a flame detection device, which is a computer, to acquire multiple target images of a monitored object captured over a time series, extract a flame candidate region from each of the target images, calculate an index indicating that the object shown in the flame candidate region is moving in a flame-like manner as the degree to which the amount of movement of a tracking point set in the flame candidate region fluctuates relative to the average value of the amount of movement, calculate the amount of movement and the average value for each of a first direction in the target image and a second direction different from the first direction, and determine whether or not a flame is shown in the flame candidate region based on the index in each direction. [Effects of the Invention]

[0009] As described above, according to the present invention, even if a fire breaks out in a moving body, it is possible to accurately determine whether or not flames are shown in an image. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing an example of the configuration of a flame detection system 1 to which a flame detection device 10 according to an embodiment is applied. [Figure 2] 1 is a block diagram showing an example of the configuration of a flame detection device 10 according to an embodiment. [Figure 3A] 3A to 3C are diagrams illustrating processing performed by a flame candidate region extraction unit 101 according to the embodiment. [Figure 3B] 3A to 3C are diagrams illustrating processing performed by a flame candidate region extraction unit 101 according to the embodiment. [Figure 4] 3 is a diagram illustrating a process performed by a flame determination unit 102 according to the embodiment. FIG. [Figure 5] 3 is a diagram illustrating a process performed by a flame determination unit 102 according to the embodiment. FIG. [Figure 6] 3 is a diagram illustrating a process performed by a flame determination unit 102 according to the embodiment. FIG. [Figure 7] 3 is a diagram illustrating a process performed by a flame determination unit 102 according to the embodiment. FIG. [Figure 8] 3 is a diagram illustrating a process performed by a flame determination unit 102 according to the embodiment. FIG. [Figure 9] 3 is a flowchart showing the flow of processing performed by the flame detection device 10 according to the embodiment. [Figure 10] 10 is a diagram illustrating a process performed by a flame determination unit 102 according to a first modified example of the embodiment. FIG. [Figure 11] 10 is a diagram illustrating a process performed by a flame determination unit 102 according to a second modification of the embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] FIG. 1 is a diagram showing an example of the configuration of a flame detection system 1 to which a flame detection device 10 according to an embodiment is applied. The flame detection system 1 includes, for example, a camera CA and the flame detection device 10. The camera CA and the flame detection device 10 are communicatively connected via a communication network, short-range communication using a wireless LAN (Local Area Network), or the like, or via a USB (Universal Serial Bus) cable or the like. The camera CA and the flame detection device 10 may also be provided integrally. In other words, the camera CA may be built into the flame detection device 10. The monitored object T is an area or object that is the target of monitoring by the flame detection system 1. The monitored object T may be set arbitrarily. For example, the monitored object T may be a roadway, a tunnel premises, a building, a building premises, etc. The camera CA captures an image of the monitoring target T. The camera CA may capture a moving image or a still image. The camera CA outputs the captured image to the flame detection device 10. Flame detection device 10 is a computer, and is realized by, for example, a PC (Personal Computer), a server device, etc. Flame detection device 10 acquires an image (target image) of a monitoring target T from a camera CA, and determines whether a flame is shown in the acquired image.

[0013] 2 is a block diagram showing an example configuration of flame detection device 10 according to an embodiment. Flame detection device 10 includes, for example, image acquisition unit 100, flame candidate region extraction unit 101, flame determination unit 102, determination result output unit 103, and image information storage unit 104. The functional units of the flame detection device 10 (image acquisition unit 100, flame candidate area extraction unit 101, flame judgment unit 102, and judgment result output unit 103) are realized by having a CPU (Central Processing Unit) that the flame detection device 10 has as hardware execute a program.

[0014] The image acquisition unit 100 acquires image information of a target image. The image acquisition unit 100 acquires image information of a target image captured by a camera CA. The image acquisition unit 100 outputs the acquired image information to the flame candidate region extraction unit 101.

[0015] The flame candidate region extraction unit 101 extracts a flame candidate region from a target image. A flame candidate region is a region in the target image that shows an object that looks like a flame. A specific method by which the flame candidate region extraction unit 101 extracts a flame candidate region from a target image will be described in detail later.

[0016] The flame determination unit 102 determines whether or not a flame is shown in the flame candidate region. The flame determination unit 102 determines whether or not a flame is shown in the flame candidate region based on the movement of the object imaged in the flame candidate region. A specific method by which the flame determination unit 102 determines whether or not a flame is shown in the flame candidate region will be described in detail later.

[0017] The determination result output unit 103 outputs the determination result by the flame determination unit 102, i.e., whether or not a flame is shown in the target image. The determination result output unit 103 may be configured to display the determination result, for example, by displaying the determination result on a display (not shown). In this case, if a flame is shown in the target image, the user may be alerted by displaying a message indicating the possibility of a fire or by outputting an alarm sound, for example.

[0018] The image information storage unit 104 stores image information. The image information is image information of a target image. The image information may include information indicating the flame candidate region extracted by the flame candidate region extraction unit 101, the determination result determined by the flame determination unit 102, the index used for the determination, and the like. The image information storage unit 104 is configured by a storage medium, such as a hard disk drive (HDD), flash memory, electrically erasable programmable read-only memory (EEPROM), random access read / write memory (RAM), read-only memory (ROM), or any combination of these storage media.

[0019] Here, we will explain how the flame candidate region extraction unit 101 extracts a flame candidate region from a target image. The flame candidate region extraction unit 101 extracts a flame candidate region from a target image through a plurality of stepwise processes.

[0020] In the first step, the flame candidate region extraction unit 101 extracts a first candidate region based on color information in the target image. Specifically, the flame candidate region extraction unit 101 binarizes pixels in the target image according to whether their pixel values ​​indicate a color that resembles a flame. For example, when color information is expressed in RGB (Red, Green, Blue), a flame-like color is a color in which the R value, which is the red component in the color information, is greater than a threshold, or a color in which the difference (RG) value between the red and green components is greater than a threshold.When color information is expressed in HSV (Hue, Saturation, Value, Brightness), a flame-like color is a color in which the H value, which is the hue component, is reddish and the V value, which is the brightness component, is greater than a threshold. The flame candidate region extraction unit 101 extracts a region consisting of a pixel group that exhibits a flame-like color from the binarized pixels, and designates the extracted region as a first candidate region. In this case, the flame candidate region extraction unit 101 may exclude, as noise, regions that have an area less than a threshold value from the regions consisting of a pixel group that exhibits a flame-like color, and designate regions that have an area equal to or greater than the threshold value as the first candidate region.

[0021] In the second stage, the flame candidate region extraction unit 101 performs a tracking process to determine whether the first candidate region extracted from a target image (hereinafter referred to as the second image) captured after the target image (hereinafter referred to as the first image) from which the first candidate region was extracted is caused by the same object.

[0022] FIG. 3 (FIGS. 3A and 3B) is a diagram illustrating the tracking process. FIG. 3A shows a schematic example of a target image TG1. In the target image TG1, regions R1 and R2 are extracted as first candidate regions. For example, a flashing red light is shown in region R1. A flame is shown in region R2. Such a red light and flame are extracted as first candidate regions because they have a flame-like color. The flame candidate region extraction unit 101 sets a pixel at a specific position in the region extracted as the first candidate region, for example, a pixel located at the center of gravity, as a tracking point, and acquires the position coordinates of the tracking point. In the example of this figure, the flame candidate region extraction unit 101 acquires the position coordinates of pixel R1G as the tracking point in region R1 and pixel R2G as the tracking point in region R2.

[0023] 3B shows a schematic example of a target image TG2. The target image TG2 is a target image captured chronologically later than the target image TG1. For example, in a moving image, the target image TG2 is an image captured in the frame following the target image TG1. In the target image TG2, region R2# is extracted as the first candidate region. The flame candidate region extraction unit 101 acquires the position coordinates of pixel R2G# as a tracking point for region R2# extracted as the first candidate region. The flame candidate region extraction unit 101 calculates the distance between the tracking point (pixel R2G) acquired in the first image and the tracking point (pixel R2G#) acquired in the second image. If the calculated distance is less than a threshold, the flame candidate region extraction unit 101 determines that the first candidate region extracted in the first image and the first candidate region extracted in the second image are caused by the same object. In this case, the flame candidate region extraction unit 101 performs tracking processing on target images captured after the second image, using region R2 as the tracking target. The method for setting the threshold may be determined appropriately, but may be set, for example, based on the fastest-moving object in the monitored object T. For example, if the monitored object T is a roadway or a tunnel premises, the threshold may be set based on the distance traveled by a vehicle traveling at the maximum speed of a vehicle traveling on the roadway or tunnel premises, or at a speed that includes a predetermined margin above the maximum speed, during the time interval between capturing the target image TG1 and the target image TG2. On the other hand, if the flame candidate region extraction unit 101 does not extract a first candidate region in the second image near the tracking point (pixel R1G) acquired in the first image, it determines that the first candidate region extracted in the first image and the first candidate region extracted in the second image are not due to the same object. In other words, it determines that the object captured in region R1 of the first image could not be tracked in the second image. In this case, it excludes region R1 from the tracking targets, and does not perform tracking processing on object images captured after the second image.

[0024] The flame candidate region extraction unit 101 determines whether to set the first candidate region as the second candidate region based on the results of the tracking process. If the first candidate region extracted in the first stage becomes the tracking target, that is, if region R2# is extracted in the second image near region R2 extracted in the first image, the flame candidate region extraction unit 101 sets region R2 and / or region R2# as the second candidate region. On the other hand, if the first candidate region extracted in the first stage is excluded from the tracking target, that is, if the first candidate region is not extracted in the second image near region R1 extracted in the first image, the flame candidate region extraction unit 101 does not set the first candidate region (region R1) as the second candidate region. In this way, by performing tracking processing in the second stage, even if an object behaving more like a flame is moving, it can be set as the second candidate region.

[0025] In this case, the flame candidate region extraction unit 101 may be configured to designate a region that can be tracked in a predetermined number or more of consecutive target images in chronological order, for example, several frames of images, as the second candidate region, and not designate a region that can no longer be tracked in several frames of images as the second candidate region.

[0026] In the third step, the flame candidate region extraction unit 101 calculates the degree to which the second candidate region resembles a flame, based on the complexity of the shape of the second candidate region and / or the time-series change in color.

[0027] First, a method will be described in which the flame candidate region extraction unit 101 calculates the degree to which the second candidate region resembles a flame based on the complexity of the shape of the second candidate region. The flame candidate region extraction unit 101 extracts, as contour information, a sequence Pn indicating the coordinates of a group of pixels that make up the contour. The sequence Pn is, for example, a sequence in which the group of pixels that make up the contour are arranged in order along a predetermined direction, for example, clockwise or counterclockwise. The flame candidate region extraction unit 101 selects three pixels that are spaced a predetermined distance apart from the sequence Pn. Specifically, the flame candidate region extraction unit 101 selects three pixels (Pk-d, Pk, Pk+d) that are spaced a distance d apart from the sequence Pn{P1, P2, ..., Pn}.

[0028] The flame candidate region extraction unit 101 calculates the angle formed by a line segment connecting each of the three selected pixels along the contour. Specifically, the flame candidate region extraction unit 101 calculates vectors V1 and V2 pointing from the origin Pk to the other points (Pk-d) and (Pk+d) among the coordinates of each of the three pixels (Pk-d, Pk, Pk+d) selected from the sequence Pn. Vector V1 is a vector pointing from pixel (Pk) to pixel (Pk-d). Vector V2 is a vector pointing from pixel (Pk) to pixel (Pk+d). The flame candidate region extraction unit 101 calculates the angle θ formed by the line segment using the dot product of the vectors, for example, as shown in equation (1).

[0029] dx1*dx2+dy1*dy2 =√(dx1^2+dy1^2)×√(dx2^2+dy2^2)×cosθ …(1) however, (dx1, dy1): Coordinate values ​​of vector V1 (dx2, dy2): Coordinate values ​​of vector V2 θ: angle between lines

[0030] In this way, the flame candidate region extraction unit 101 calculates the angle of the contour line as the angle formed by the line segment connecting three pixels selected from the group of pixels constituting the contour of the flame candidate region R along the contour. For example, the flame candidate region extraction unit 101 calculates the angle of the contour line as the angle formed by each line segment starting from each of the pixels constituting the contour. The flame candidate region extraction unit 101 calculates the degree of flame-likeness based on the complexity of the shape using statistics of the angles formed by the contour line. The statistics here may be any statistics that serve as an indicator of whether the angles formed by the contour line are concentrated at a specific angle, such as 180 degrees. For example, the statistics may be the mean value, representative value, mode, maximum value, minimum value, variance, standard deviation, or a combination thereof.

[0031] Here, a specific method for calculating the indices using statistics by the flame candidate region extraction unit 101 will be described. In this embodiment, two indices are used to indicate the complexity of the flame candidate region shape: (1) the concentration level, and (2) the variation in the angle that becomes the most frequent value. (1) The concentration index is an index indicating the degree to which angles formed by a contour line are concentrated at a specific angle, and is the ratio of the number of angles that form the most frequent value to the total number of angles formed by the contour line. For example, if line segments are generated starting from each of the pixels that form the contour and the angles formed by each of the generated line segments are obtained, the total number of angles formed by the contour line will be the same as the total number of pixels that form the contour line. In this case, if the concentration index calculated from the second candidate region is less than a threshold, the flame candidate region extraction unit 101 determines that the object captured in the second candidate region is likely to be a flame. On the other hand, if the concentration index calculated from the second candidate region is equal to or greater than the threshold, the flame candidate region extraction unit 101 determines that the object captured in the second candidate region is not a flame.

[0032] (2) The variance in the angle resulting from the mode is an index indicating the degree to which the mode values ​​of the angles formed by the contour lines in each image are not concentrated, i.e., the degree of variance, when using video images of the monitored object T. For example, the flame candidate region extraction unit 101 calculates the mode value for each target image constituting the video. The mode value here is the angle that is the most frequent among the angles formed by the contour lines in the flame candidate regions extracted from the target images. If the variance in the mode values ​​calculated from each of the second candidate regions that are consecutive in time series is large and equal to or greater than a threshold, the flame candidate region extraction unit 101 determines that the object captured in the second candidate region is likely to be a flame. On the other hand, if the variance in the mode values ​​calculated from each of the second candidate regions is small and less than the threshold, the flame candidate region extraction unit 101 determines that the object captured in the second candidate region is not a flame.

[0033] Next, a method will be described in which the flame candidate region extraction unit 101 calculates the degree to which the second candidate region resembles a flame based on the time-series change in color in the second candidate region. For example, the flame candidate region extraction unit 101 acquires the pixel values ​​of each of the pixel groups constituting the second candidate region shown in each of the multiple target images captured in time series. The flame candidate region extraction unit 101 calculates the rhythm (frequency characteristics) of changes in the pixel values ​​by performing frequency analysis, for example, FFT (Fast Fourier Transform), on the time series changes of a representative pixel value selected from the acquired pixel values, such as an average value. The flame candidate region extraction unit 101 calculates the degree to which the rhythm (frequency characteristics) of pixel value changes resembles the rhythm of flame color changes as the degree to which the second candidate region resembles a flame.

[0034] Fig. 4 is a diagram explaining a method for calculating the degree of flame-likeness of pixels that make up the second candidate region. Fig. 4 shows frequency characteristics calculated by performing an FFT on pixel values. The horizontal axis of the graph shown in Fig. 4 represents frequency, and the vertical axis represents signal strength.

[0035] The graph on the left side of Figure 4 shows the frequency characteristics of pixel values ​​in the area where a flame is displayed in a video. In this figure, the spectrum (signal strength) is greatest in the range of 1.0 [Hz] < frequency f ≦ 4.5 [Hz]. This shows that when a burning flame is captured on video, the color changes rhythmically at approximately 1 to 4.5 times per second.

[0036] The graph on the right side of Figure 4 shows the frequency characteristics of pixel values ​​in the area of ​​a moving image showing a car passing by. In this figure, the spectrum (signal strength) is strongest in the frequency range below 1.5 Hz. This shows that when a moving image of a car passing by is captured, the color changes at a rhythm of approximately 1.5 times per second or less.

[0037] If the frequency characteristics of the pixels based on the pixel group constituting the second candidate region have a peak, i.e., the greatest signal strength, in a predetermined frequency range, for example, 1.0 [Hz] < frequency f ≦ 4.5 [Hz], the flame candidate region extraction unit 101 determines that the object imaged in the second candidate region is likely to be a flame. On the other hand, if the frequency characteristics of the pixels based on the pixel group constituting the second candidate region do not have a peak in the predetermined frequency range, the flame candidate region extraction unit 101 determines that the object imaged in the second candidate region is not a flame. The flame candidate region extraction unit 101 determines, as a flame candidate region, a second candidate region that has a high degree of flame-likeness and in which the object imaged in the second candidate region is determined to be a flame.

[0038] Here, a method for determining whether or not a flame is shown in a flame candidate region by the flame determination unit 102 will be described. The flame determination unit 102 determines whether or not a flame is shown in a flame candidate region based on the movement of an object captured in the flame candidate region.

[0039] The flame determination unit 102 divides the flame candidate region into grid-like regions and sets a tracking point in each of the divided regions. The flame determination unit 102 calculates the optical flow at each tracking point. Here, optical flow is a concept that represents the movement of an object captured in an image, for example, a concept that represents the direction in which a flame captured in an image moves. Here, the flame determination unit 102 acquires the color change at each tracking point set in each grid-like region obtained by dividing the flame candidate region. The flame determination unit 102 performs a calculation related to the color change at the tracking point. For example, if the color changes from a first point to a second point, the flame determination unit 102 calculates a vector from the first point to the second point. The flame determination unit 102 performs this calculation for each of the multiple tracking points described above to obtain multiple vectors. The flame determination unit 102 then combines the multiple vectors obtained in this way to calculate a vector that indicates the direction and size of the flame's movement as the optical flow.

[0040] The flame determination unit 102 calculates an index indicating whether or not an object captured in a flame candidate region is moving like a flame, based on the time-series change in the optical flow.

[0041] Generally, a flame behaves as if it is gradually spreading while repeatedly growing and shrinking, i.e., fluctuating. On the other hand, a moving object such as a vehicle generally moves in a certain direction at a constant speed or while changing its speed. Taking advantage of this characteristic movement of a flame, in this embodiment, the flame determination unit 102 calculates an index indicating whether an object captured in an image of a flame candidate region is moving like a flame, based on the time-series change in the optical flow.

[0042] Fig. 5 is a diagram showing an example of optical flow calculated from a video image of a flame. Fig. 5 shows frame images from time t = 0 to time t = 11 in chronological order. Each frame image shows a flickering flame, and arrows indicate the optical flow calculated in the flame candidate region (the region where the flame is imaged) extracted from each frame image. As shown in this example, when a flame is imaged, the optical flow calculated in each frame image shows various movements and does not move in a fixed direction like a moving object such as a vehicle.

[0043] The flame determination unit 102 extracts, in time series, the values ​​of the x-axis direction component (hereinafter referred to as x-component value) and the y-axis direction component (hereinafter referred to as y-component value) of the optical flow calculated for each tracking point. Furthermore, the flame determination unit 102 calculates a simple arithmetic average value of the x-component values ​​in a certain time interval (hereinafter referred to as x-average value) based on the time-series change in the x-component values ​​in the optical flow. Furthermore, the flame determination unit 102 calculates a simple arithmetic average value of the y-component values ​​in a certain time interval (hereinafter referred to as y-average value) based on the time-series change in the y-component values ​​in the optical flow.

[0044] The x-axis and y-axis directions may be determined arbitrarily. For example, the x-axis direction is the horizontal direction (lateral direction) in the target image, and the y-axis direction is the vertical direction (longitudinal direction) in the target image. Alternatively, for example, the direction of the first principal component obtained by performing principal component analysis on the horizontal and vertical components in the optical flow image may be used as the x-axis and the direction of the second principal component as the y-axis. Alternatively, if the monitoring target T includes a road, the traffic direction on the road may be used as the x-axis, and the direction perpendicular to the traffic direction may be used as the y-axis.

[0045] Fig. 6 is a diagram showing an example of time-series changes in the x component value in optical flow and the x average value. The horizontal axis in Fig. 6 represents time, and the vertical axis represents the component value. As shown in the example in this diagram, when a flame repeatedly grows larger and smaller, the x component value repeatedly falls below and exceeds the x average value.

[0046] The flame determination unit 102 calculates the degree to which the x component value fluctuates with respect to the x average value as an index showing whether or not there is flame-like movement. Here, the x component value fluctuating with respect to the x average value means that the x component value crosses the x average value. More specifically, this means that the x component value changes from a state where it is less than the x average value to a state where it is equal to or greater than the x average value, or that the x component value changes from a state where it is equal to or greater than the x average value to a state where it is less than the x average value. The number of times the x component value crosses the x average value (hereinafter referred to as the number of crosses) can be used as the degree of such fluctuation.

[0047] Similarly, for the y direction, the flame determination unit 102 calculates the degree to which the y-component value fluctuates relative to the y-average value as an index indicating whether or not the flame-like movement is occurring. Here, the y-component value fluctuating relative to the y-average value means that the y-component value crosses the y-average value. More specifically, this means that the y-component value changes from being less than the y-average value to being equal to or greater than the x-average value, or that the y-component value changes from being equal to or greater than the y-average value to being less than the y-average value. The number of times the y-component value crosses the y-average value (hereinafter referred to as the number of crosses) can be used as the degree of such fluctuation. Note that hysteresis may be applied to the determination of whether or not the crossing has occurred. For example, the crossing may be determined to have occurred when the x-component value changes from being less than the x-average value to being equal to or greater than (x-average value + predetermined value α), or when the x-component value changes from being equal to or greater than the x-average value to being less than (x-average value - predetermined value β). Here, the predetermined values ​​α and β may be the same or different.

[0048] Fig. 7 is a diagram showing an example of time-series changes in the x- and y-component values ​​of the optical flow in a flame candidate region where a flame is captured. The horizontal axis of Fig. 7 represents time, and the vertical axis represents the component value. Avex represents the x-average value. Avey represents the y-average value. In the example shown in this diagram, the number of crossings in the x direction is 8, and the number of crossings in the y direction is 6. The total number of crossings, which is the number of crossings in the x direction and the y direction, is 14.

[0049] Fig. 8 is a diagram showing an example of time-series changes in the x- and y-component values ​​of the optical flow in a flame candidate region in which a passing vehicle is captured. The horizontal axis of Fig. 8 represents time, and the vertical axis represents the component value. Avex represents the x-average value. Avey represents the y-average value. In the example shown in this diagram, the number of crossings in the x direction is 3, and the number of crossings in the y direction is 2. The total number of crossings, which is the number of crossings in the x direction and the y direction, is 5.

[0050] The flame determination unit 102 calculates this number of crosses as an index showing the degree of oscillation, that is, whether or not there is a flame-like movement. For example, if the number of crosses calculated from a flame candidate region is equal to or greater than a threshold, the flame determination unit 102 determines that a flame is shown in the flame candidate region. On the other hand, if the number of crosses calculated from a flame candidate region is less than the threshold, the flame determination unit 102 determines that a flame is not shown in the flame candidate region.

[0051] The flame determination unit 102 may determine whether or not a flame is shown in the flame candidate region using only either the number of crosses in the x direction or the number of crosses in the y direction. The flame determination unit 102 may also determine whether or not a flame is shown in the flame candidate region using only either the number of times the x component value changes from a state where it is less than the x average value to equal to or greater than the x average value (first cross number) or the number of times the x component value changes from a state where it is equal to or greater than the x average value to less than the x average value (second cross number).

[0052] Here, the flow of processing performed by the flame detection device 10 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of processing performed by the flame detection device 10 according to the embodiment.

[0053] The flame detection device 10 acquires time-series successive target images (frame images) (step S10). The flame detection device 10 acquires, for example, moving images of the monitoring target T captured by the camera CA as time-series successive target images. Next, the flame detection device 10 extracts a first candidate region from each target image (step S11). Based on the color information in each target image, the flame detection device 10 extracts a surface region that shows a flame-like color as the first candidate region. Next, flame detection device 10 determines whether to designate the first candidate region as a second candidate region based on the results of the tracking process of the first candidate region (step S12). If the first candidate region extracted in a frame image can be tracked over several frames from the next frame onwards, flame detection device 10 designates the first candidate region as a second candidate region. On the other hand, if the first candidate region extracted in a frame image cannot be tracked over several frames from the next frame onwards, flame detection device 10 does not designate the first candidate region as a second candidate region.

[0054] Next, the flame detection device 10 determines whether or not to set the second candidate region as a flame candidate region based on the characteristics of the second candidate region, that is, the complexity of the outline shape and / or the time-series change in color (step S13). For example, flame detection device 10 designates a second candidate region as a flame candidate region if the angles formed by the contour shape of the second candidate region are not concentrated at a specific angle. On the other hand, flame detection device 10 does not designate a second candidate region as a flame candidate region if the angles formed by the contour shape of the second candidate region are concentrated at a specific angle. Alternatively, flame detection device 10 designates a second candidate region as a flame candidate region if the most frequent value of the angles formed by the contour shape of the second candidate region varies over time. On the other hand, flame detection device 10 does not designate a second candidate region as a flame candidate region if the most frequent value of the angles formed by the contour shape of the second candidate region does not vary over time. For example, flame detection device 10 calculates the frequency characteristics of pixel values ​​in the second candidate region, and if the spectrum of the pixel values ​​resembles the spectrum of a flame, designates the second candidate region as a flame candidate region. On the other hand, if the spectrum of the pixel values ​​in the second candidate region does not resemble the spectrum of a flame, flame detection device 10 does not designate the second candidate region as a flame candidate region.

[0055] Next, the flame detection device 10 calculates the optical flow in the flame candidate region (step S14). The flame detection device 10 calculates vectors indicating the movement of each of the multiple tracking points set in the flame candidate region, and calculates the optical flow by combining the calculated vectors.

[0056] Then, flame detection device 10 determines whether a flame is indicated in the flame candidate region based on the degree to which the optical flow components fluctuate relative to the average value (number of crosses) (step S15). Flame detection device 10 calculates the number of crosses for each of the x and y components of the optical flow, and determines the degree of fluctuation (number of crosses) as the sum of the number of crosses for the x component and the number of crosses for the y component. If the number of crosses is equal to or greater than a threshold, flame detection device 10 determines that a flame is indicated in the flame candidate region. On the other hand, if the number of crosses is less than the threshold, flame detection device 10 determines that a flame is not indicated in the flame candidate region.

[0057] As described above, the flame detection device 10 of the embodiment includes an image acquisition unit 100, a flame candidate region extraction unit 101, and a flame determination unit 102. The image acquisition unit 100 acquires moving images. The moving images are a plurality of target images of the monitoring target T captured in time series. The flame candidate region extraction unit 101 extracts a flame candidate region from each of the target images. The flame determination unit 102 calculates an index that indicates the flame-like movement of the target shown in the flame candidate region. The flame determination unit 102 calculates the index using the degree to which the movement amount of a tracking point set in the flame candidate region fluctuates relative to the average value of that movement amount. The flame determination unit 102 determines whether a flame is shown in the flame candidate region based on the calculated index.

[0058] As a result, the flame detection device 10 of the embodiment can determine whether an object captured in an image of a flame candidate region is a flame or not, depending on whether the object captured in the image of the flame candidate region exhibits flame-like movement. Therefore, even if a fire breaks out in a moving object, it can accurately determine whether a flame is displayed in the image.

[0059] Furthermore, in the flame detection device 10 of the embodiment, the flame determination unit 102 calculates the optical flow of pixels included in the flame candidate region as the amount of movement (e.g., x component value, y component value). As a result, the flame detection device 10 of the embodiment can quantitatively calculate the movement of an object imaged in the flame candidate region using the optical flow.

[0060] Furthermore, in the flame detection device 10 of the embodiment, the flame determination unit 102 calculates an index using at least one of the first crossing count, at which the amount of movement changes from a value below the average value to a value above it, or the second crossing count, at which the amount of movement changes from a value above the average value to a value below it. As a result, in the tenth embodiment, it is possible to easily determine whether an object imaged in a flame candidate region is moving like a flame using the crossing count.

[0061] Furthermore, in the flame detection device 10 of this embodiment, the flame determination unit 102 calculates the amount of movement and the average value for each of the X-axis direction (first direction) and the y-axis direction (second direction) in the target image. The flame determination unit 102 determines whether or not a flame is shown in the flame candidate region based on the calculated number of crosses (index) in each direction. As a result, in the tenth embodiment, a vector indicating movement, for example, each component of optical flow, can be used to make an accurate determination based on the degree of fluctuation in movement in each direction.

[0062] Furthermore, in the flame detection device 10 of this embodiment, the flame candidate region extraction unit 101 extracts a first candidate region based on color information contained in multiple target images. The flame candidate region extraction unit 101 determines whether a first candidate region extracted from a first image and a first candidate region extracted from a second image captured after the first image are caused by the same object. If the first candidate region extracted from the first image and the first candidate region extracted from the second image are caused by the same object, the flame candidate region extraction unit 101 designates the first candidate region extracted from the first image and / or the first candidate region extracted from the second image as a second candidate region. The flame candidate region extraction unit 101 determines a flame candidate region from the second candidate region. Furthermore, in the embodiment of the flame detection device 10, the flame candidate region extraction unit 101 calculates the degree to which the second candidate region resembles a flame based on the frequency characteristics at which the color in the second candidate region changes, and determines whether or not to treat the second candidate region as the flame candidate region based on the calculated degree to which it resembles a flame. As a result, in the embodiment of the flame detection device 10, areas whose color and the way that color changes resemble a flame can be identified as flame candidate areas based on the color of the object captured in the target image and the rhythm of that color change.

[0063] Here, a first modification of the embodiment will be described. This modification differs from the above-described embodiment in that a regression line is used for the average values ​​(x average value and y average value). The regression line is a line that best fits the distribution of the component values ​​(x component value or y component value). Any method in the prior art can be used to determine the regression line. For example, the least squares method can be used as a method for optimally calculating the coefficients and intercepts of the regression line.

[0064] FIG. 10 is a diagram illustrating the process performed by the flame determination unit 102 according to the first modification of the embodiment. FIG. 10 shows an example in which the time series change in the x component value in the optical flow and the regression line of the x component value is used as the x average value. As shown in the example in this figure, there may be cases in which the x component value gradually decreases while fluctuating up and down. In such cases, if the number of crossings is calculated based on the simple arithmetic average value, it is difficult to appropriately obtain the degree of fluctuation. To address this issue, in this modified example, the regression line of the x component value is used as the x average value. This makes it possible to appropriately calculate the degree to which the x component value fluctuates, even when the x component value fluctuates up and down and gradually decreases (or gradually increases).

[0065] Here, a second modification of the embodiment will be described. This modification differs from the above-described embodiment in that a simple arithmetic average of preceding and following values ​​(hereinafter referred to as a preceding and following average) is used as the average value (x average value and y average value). The preceding and following average value is a value obtained by simply averaging the preceding and following values ​​of a component value (x component value or y component value). For example, the preceding and following average value is a value obtained by simply averaging the x component values ​​of several frames before and after the time at which the average value is calculated.

[0066] FIG. 11 is a diagram illustrating the process performed by the flame determination unit 102 according to the second modification of the embodiment. FIG. 11 shows an example of time-series changes in x-component values ​​in optical flow, and an example in which the average value of the x-component values ​​before and after the x-component values ​​is used as the x-average value. As shown in the example in this figure, there may be cases in which the degree of decrease is not constant as the x-component value fluctuates up and down and gradually decreases. In such cases, it is difficult to properly obtain the degree of fluctuation when the number of crossings is calculated based on a simple arithmetic average value or a regression line. To address this issue, in this modified example, the average value of the x-component values ​​before and after the x-component values ​​is used as the x-average value. As a result, even if the degree of decrease of the x component value is not constant as it fluctuates up and down and gradually decreases (or gradually increases), the x average value can be set based on nearby x component values, and the degree of fluctuation of the x component value can be calculated appropriately.

[0067] In the above-described embodiment, an example of a tracking processing method has been described, but the present invention is not limited to this method. Specifically, when the distance between the tracking point (pixel R2G) acquired in the first image and the tracking point (pixel R2G#) acquired in the second image in FIG. 3 (FIGS. 3A and 3B) is less than a threshold, it is determined that the first candidate area extracted in the first image and the first candidate area extracted in the second image are due to the same object. However, the determination of whether the area R2 and the area R2# are due to the same object is not limited to this. For example, when the tracking point (pixel R2G) in area R2 is included in area R2#, it may be determined that the first candidate area extracted in the first image and the first candidate area extracted in the second image are due to the same object. Furthermore, it may be determined that the first candidate area extracted in the first image and the first candidate area extracted in the second image are due to the same object when the tracking point (pixel R2G#) in area R2# is included in area R2.

[0068] All or part of the flame detection device 10 in the above-described embodiment may be implemented by a computer. In this case, a program for implementing the functions may be recorded on a computer-readable recording medium, and the program may be loaded and executed by a computer system. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within a computer system serving as a server or client. The program may also be designed to implement some of the functions described above, or may be capable of implementing the functions in combination with a program already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA.

[0069] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0070] 1...Flame detection system 10...Flame detection device 100...Image acquisition unit (acquisition unit) 101...Flame candidate region extraction section (extraction section) 102...Flame judgment section (judgment section)

Claims

1. an acquisition unit that acquires a plurality of target images of a monitoring target captured in time series; an extraction unit that extracts a flame candidate region from each of the target images acquired by the acquisition unit; a determination unit that determines whether a flame is shown in the flame candidate region based on an index indicating that an object shown in the flame candidate region is moving like a flame, the index being calculated using at least one of a first crossing count at which the amount of movement of a tracking point set in the flame candidate region changes from a value below the average value of the amount of movement to a value above the average value, or a second crossing count at which the amount of movement changes from a value above the average value to a value below the average value; A flame detection device comprising:

2. an acquisition unit that acquires a plurality of target images of a monitoring target captured in time series; an extraction unit that extracts a flame candidate region from each of the target images acquired by the acquisition unit; a determination unit that calculates an index indicating that the object shown in the flame candidate region is moving like a flame by calculating the degree to which the amount of movement of a tracking point set in the flame candidate region fluctuates relative to an average value of the amount of movement in a first direction and a second direction different from the first direction in the target image, and determines whether a flame is shown in the flame candidate region based on the index in each direction; A flame detection device comprising:

3. the determination unit calculates an optical flow of pixels included in the flame candidate region as the movement amount; 10. The flame detection device of claim 1.

4. the extraction unit extracts a first candidate area based on color information included in the plurality of target images; the first candidate region in the first image extracted from the first image of the plurality of target images; the first candidate area in the second image extracted from the second image captured after the first image among the plurality of target images, determining whether the first candidate area in the first image and the first candidate area in the second image are due to the same object, and if they are due to the same object, determining the first candidate area in the first image and / or the first candidate area in the second image as a second candidate area; determining the flame candidate region from the second candidate region; A flame detection device according to any one of claims 1 to 3.

5. the extraction unit calculates a degree to which the second candidate region resembles a flame based on frequency characteristics at which color in the second candidate region changes, and determines whether or not to set the second candidate region as the flame candidate region based on the calculated degree to which the second candidate region resembles a flame.

5. The flame detection device of claim 4.

6. A flame detection method performed by a flame detection device that is a computer, comprising: The acquisition unit acquires a plurality of target images in which the monitoring target is captured in chronological order, an extraction unit extracting a flame candidate region from each of the target images acquired by the acquisition unit; a determination unit determines whether a flame is shown in the flame candidate region based on an index indicating that the object shown in the flame candidate region is moving like a flame, the index being calculated using at least one of a first crossing count at which the amount of movement of the tracking point set in the flame candidate region changes from a value below the average value of the amount of movement to a value above the average value, or a second crossing count at which the amount of movement changes from a value above the average value to a value below the average value; Flame detection methods.

7. A flame detection method performed by a flame detection device that is a computer, comprising: The acquisition unit acquires a plurality of target images in which the monitoring target is captured in chronological order, an extraction unit extracting a flame candidate region from each of the target images acquired by the acquisition unit; a determination unit calculates an index indicating that the object shown in the flame candidate region is moving like a flame by calculating the degree to which the amount of movement of a tracking point set in the flame candidate region fluctuates relative to an average value of the amount of movement in a first direction and a second direction different from the first direction in the target image, and determines whether a flame is shown in the flame candidate region based on the index in each direction; Flame detection methods.

8. The flame detection device is a computer. Acquire a plurality of target images of the monitored object taken in time series, extracting flame candidate regions from each of the target images; a determination as to whether a flame is shown in the flame candidate region based on an index indicating that the object shown in the flame candidate region is moving like a flame, the index being calculated using at least one of a first crossing count at which the amount of movement of the tracking point set in the flame candidate region changes from a value below the average value of the amount of movement to a value above the average value, or a second crossing count at which the amount of movement changes from a value above the average value to a value below the average value; program.

9. The flame detection device is a computer. Acquire a plurality of target images of the monitored object taken in time series, extracting flame candidate regions from each of the target images; an index indicating that the object shown in the flame candidate region is moving like a flame is calculated as the degree to which the amount of movement of a tracking point set in the flame candidate region fluctuates relative to the average value of the amount of movement, and the amount of movement and the average value are calculated for each of a first direction in the target image and a second direction different from the first direction, and whether or not a flame is shown in the flame candidate region is determined based on the index in each direction; program.

Images