Fluid image processing device, fluid image processing system, program, and fluid image processing method

Abstract

The present invention provides a fluid image processing device that makes it possible to suppress erroneous detection of a fluid or the like which is not a target to be monitored.　A gas image processing device 1 comprises: a first image processing unit 11 that performs frequency processing for extracting a gas signal with respect to a first time-series image, which has been captured using light of a first wavelength corresponding to a gas to be monitored, to obtain a gas candidate region; a second image processing unit 12 that performs frequency processing for extracting a gas exclusion signal with respect to a second time-series image, which has been captured using light of a second wavelength differing from the first wavelength, to obtain a gas exclusion region; and a computation unit 13 that obtains a gas region on the basis of the gas candidate region and the gas exclusion region.

Description

Fluid image processing device, fluid image processing system, program, and fluid image processing method

[0001] The present invention relates to a fluid image processing device, a fluid image processing system, a program, and a fluid image processing method for detecting leakage of a fluid such as gas from time-series images.

[0002] A technology has been proposed to detect leaks of gas or other fluids using infrared images taken with an infrared camera of objects including surveillance targets. However, if the objects include steam, people, automobiles, or other moving objects, these may be mistakenly detected as leaks of gas or other fluids.

[0003] In response to this, a technology has been proposed for identifying suspicious objects and the like from the extracted difference between a visible image acquired by a visible camera and an infrared image acquired by an infrared camera (see, for example, Patent Document 1).

[0004] Patent No. 5991224

[0005] However, when trying to detect fluids by comparing visible and infrared images, there is a possibility that faint noise such as steam may be mistaken for a leak of fluid such as gas.

[0006] The present invention has been made to solve these problems, and aims to provide a fluid image processing device that can suppress false detection of fluids that are not the target of monitoring, etc. The present invention also aims to provide a fluid image processing system including the fluid image processing device, a program executed by the fluid image processing device, and a fluid image processing method.

[0007] In order to solve the above-mentioned problems, the present invention provides a fluid image processing device that includes a first image processing unit that performs frequency processing to extract the monitoring target from a first time-series image taken with light of a first wavelength corresponding to the fluid to be monitored, and obtains a candidate monitoring target area, a second image processing unit that performs frequency processing to extract the monitoring target from a second time-series image taken with light of a second wavelength different from the first wavelength, and obtains a monitoring target exclusion area, and a calculation unit that obtains the monitoring target based on the candidate monitoring target area and the monitoring target exclusion area.

[0008] The present invention also provides a fluid image processing system that includes a first image acquisition unit that acquires first time series images taken with light of a first wavelength corresponding to the fluid to be monitored, a second image acquisition unit that acquires second time series images taken with light of a second wavelength different from the first wavelength, and a fluid image processing device that performs image processing on the first time series images and the second time series images, wherein the fluid image processing device includes a first image processing unit that performs frequency processing on the first time series images to extract the monitoring target and acquire a candidate monitoring target area, a second image processing unit that performs frequency processing on the second time series images to extract the monitoring target and acquire a monitoring target exclusion area, and a calculation unit that acquires the monitoring target based on the candidate monitoring target area and the monitoring target exclusion area.

[0009] Furthermore, the present invention is a program executed by a fluid image processing device that performs image processing on time-series images, which program causes a first image processing device to perform frequency processing on first time-series images captured with light of a first wavelength corresponding to the fluid to be monitored to extract the monitoring target and obtain a candidate monitoring target area, causes a second image processing unit to perform frequency processing on second time-series images captured with light of a second wavelength different from the first wavelength to extract the monitoring target and obtain a monitoring target exclusion area, and causes a calculation unit to perform a step of obtaining the monitoring target based on the monitoring target candidate area and the monitoring target exclusion area.

[0010] The present invention also provides a fluid image processing method that performs the steps of: performing frequency processing on a first time series image taken with light of a first wavelength corresponding to the fluid to be monitored to extract the monitoring target and obtain a candidate monitoring target area; performing frequency processing on a second time series image taken with light of a second wavelength different from the first wavelength to extract the monitoring target and obtain a monitoring target exclusion area; and obtaining the monitoring target based on the candidate monitoring target area and the monitoring target exclusion area.

[0011] According to the present invention, a subject is photographed using light of different wavelengths corresponding to the fluid being monitored, and time-series images are acquired. These time-series images are subjected to frequency processing to extract the monitoring target, and the monitoring target is identified based on the candidate monitoring target area and the excluded monitoring target area. This makes it possible to suppress false detection of fluids that are not the monitoring target.

[0012] 1 is a functional block diagram showing an example of a gas image processing device and a gas image processing system according to the present embodiment; FIG. 2 is a flowchart showing an example of a gas image processing method according to the present embodiment; FIG. 3 is an explanatory diagram showing an example of a time-series infrared image; FIG. 4 is an explanatory diagram showing an example of a time-series infrared image; FIG. 5 is an explanatory diagram showing an example of a time-series infrared image; FIG. 6 is an explanatory diagram showing an example of a time-series infrared image; FIG. 7 is an explanatory diagram showing an example of a first frequency-processed image from which a gas signal has been extracted; FIG. 8 is an explanatory diagram showing an example of a first frequency-processed image from which a gas signal has been extracted; FIG. 9 is an explanatory diagram showing an example of a first frequency-processed image from which a gas signal has been extracted; FIG. 10 is an explanatory diagram showing an example of a first frequency-processed image from which a gas signal has been extracted; FIG. 11 is an explanatory diagram showing an example of a first frequency-processed image where the gas signal has a maximum value; FIG. 12 is an explanatory diagram showing an example of a gas candidate region; FIG. 13 is an explanatory diagram showing an example of a time-series visible image; FIG. 14 is an explanatory diagram showing an example of a time-series visible image; FIG. 15 is an explanatory diagram showing an example of a time-series visible image; FIG. 16 is an explanatory diagram showing an example of a second frequency-processed image from which a gas exclusion signal has been extracted. FIG. 1 is an explanatory diagram showing an example of a second frequency processed image from which a gas exclusion signal is extracted. FIG. 2 is an explanatory diagram showing an example of a second frequency processed image from which a gas exclusion signal is extracted. FIG. 3 is an explanatory diagram showing an example of a second frequency processed image from which a gas exclusion signal is extracted. FIG. 4 is an explanatory diagram showing an example of a gas exclusion region. FIG. 5 is an explanatory diagram showing a first embodiment of a gas region determination method. FIG. 6 is an explanatory diagram showing a second embodiment of a gas region determination method. FIG. 7 is an explanatory diagram showing a fourth embodiment of a gas region determination method. FIG. 8 is an explanatory diagram showing a fifth embodiment of a gas region determination method. FIG. 9 is an explanatory diagram showing a fifth embodiment of a gas region determination method.

[0013] Hereinafter, with reference to the drawings, an example of a gas image processing device, which is an embodiment of the fluid image processing device of the present invention, will be described. Also, an example of a gas image processing system, which is an embodiment of the fluid image processing system of the present invention, will be described. Furthermore, an example of a program executed by the gas image processing device will be described. Also, an example of a gas image processing method, which is an embodiment of the fluid image processing method of the present invention, will be described.

[0014] <Configuration Example of Gas Image Processing Apparatus and Gas Image Processing System> FIG. 1 is a functional block diagram showing an example of a gas image processing apparatus and a gas image processing system according to the present embodiment.

[0015] Gas image processing device 1 includes first image processing unit 11, second image processing unit 12, and calculation unit 13. Gas image processing system 100 includes first image acquisition unit 101, second image acquisition unit 102, gas image processing device 1, and information processing device 200.

[0016] The gas image processing device 1 is connected to a first image acquisition unit 101 and a second image acquisition unit 102. The first image acquisition unit 101 is connected to a first image processing unit 11. The second image acquisition unit 102 is connected to a second image processing unit 12.

[0017] The first image acquisition unit 101 is a camera equipped with an optical system, a filter, an image sensor, a signal processing unit, and the like (not shown). The first image acquisition unit 101 is configured to pass only light of a first wavelength corresponding to the monitored object through a filter and receive the light with an image sensor. The first wavelength is, for example, infrared light. If the monitored object is, for example, methane, a filter that passes a wavelength band from 3.2 μm to 3.4 μm is used. The first image acquisition unit 101 captures time-series still images of infrared images, video images of infrared images, etc. of a subject including the monitored object, and acquires a first time-series image MD1. The gas image processing device 1 may be equipped with the first image acquisition unit 101.

[0018] The second image acquisition unit 102 is a camera equipped with an optical system, a filter, an image sensor, a signal processing unit, and the like (not shown). The second image acquisition unit 102 is configured to pass only light of a second wavelength corresponding to the monitored object through a filter and receive the light with an image sensor. The second wavelength is, for example, visible light, different from the first wavelength. The second image acquisition unit 102 captures time-series still images of visible images, video images of visible images, and the like of a subject including the monitored object, and acquires second time-series images MD2. The gas image processing device 1 may include a first image acquisition unit 101 and a second image acquisition unit 102.

[0019] The first image processing unit 11, the second image processing unit 12, and the calculation unit 13 are configured by a CPU (Central Processing Unit) or the like. The first image processing unit 11, the second image processing unit 12, and the calculation unit 13 may be realized by processing by a DSP (Digital Signal Processor). The first image processing unit 11, the second image processing unit 12, and the calculation unit 13 may be realized in part by processing by a DSP. The first image processing unit 11, the second image processing unit 12, and the calculation unit 13 may be realized in whole by processing by a DSP.

[0020] The first image processing unit 11 performs frequency processing to extract gas from the first time-series image MD1 to obtain a first frequency-processed image MD10. The first image processing unit 11 also obtains a gas candidate area CL11, which is an example of a monitoring target candidate area, from the first frequency-processed image MD10.

[0021] The second image processing unit 12 performs frequency processing to extract gas from the second time-series image MD2 to obtain a second frequency-processed image MD20. The second image processing unit 12 also obtains a gas exclusion region CL21, which is an example of a monitoring exclusion region, from the second frequency-processed image MD20.

[0022] The calculation unit 13 obtains the gas region Eg based on the gas candidate region CL11 and the gas excluded region CL21.

[0023] The information processing device 200 is a personal computer, a smartphone, a tablet terminal, etc. The information processing device 200 includes a control unit 201 and an output unit 202.

[0024] The control unit 201 is configured with a CPU or the like. The control unit 201 may also be realized by processing using a DSP. The output unit 202 is configured with a display or the like. The output unit 202 is, for example, a liquid crystal display. The output unit 202 may also be an organic light emitting diode display (OLED display), a plasma display, or the like, instead of a liquid crystal display.

[0025] The information processing device 200 receives the first time-series image MD1, the first frequency-processed image MD10, and the gas candidate region CL11 acquired by the gas image processing device 1. The control unit 201 displays the first time-series image MD1, the first frequency-processed image MD10, and the gas candidate region CL11 on the output unit 202.

[0026] The information processing device 200 also receives the second time-series image MD2, the second frequency-processed image MD20, and the gas exclusion region CL21 acquired by the gas image processing device 1. The control unit 201 displays the second time-series image MD2, the second frequency-processed image MD20, and the gas exclusion region CL21 on the output unit 202.

[0027] The information processing device 200 also receives the gas region Eg acquired by the gas image processing device 1. The control unit 201 displays the gas region Eg on the output unit 202.

[0028] <Example of Program Functions> The first image processing unit 11, the second image processing unit 12, and the calculation unit 13 execute the programs to realize the above-described functions.

[0029] The program causes the first image processing unit 11 to execute a step of performing frequency processing to extract gas from the first time-series image MD1. The program also causes the first image processing unit 11 to execute a step of acquiring a gas candidate region CL11 from the first frequency-processed image MD10.

[0030] Furthermore, the program causes the second image processing unit 12 to execute a step of performing frequency processing to extract gas from the second time-series image MD2. The program also causes the second image processing unit 12 to execute a step of acquiring a gas exclusion region CL21 from the second frequency-processed image MD20.

[0031] Furthermore, the program causes the calculation unit 13 to execute a step of acquiring the gas region Eg based on the gas candidate region CL11 and the gas excluded region CL21.

[0032] The program is stored in a RAM (Random Access Memory), a ROM (Read Only Memory), or the like. The program may also be stored in a HDD (Hard Disk Drive), or the like. Furthermore, the program may be provided stored in a storage medium. The storage medium may be an external storage medium such as a magnetic disk, an optical disk, or an SSD (Solid State Drive).

[0033] Furthermore, by updating the program, a function for acquiring a gas exclusion region from a visible image and a function for acquiring a gas region based on a gas candidate region and a gas exclusion region may be added. Furthermore, the first image processing unit 11, the second image processing unit 12, and the calculation unit 13 may be realized in part or in whole by processing using dedicated hardware circuits.

[0034] 2 is a flowchart showing an example of a gas image processing method according to the present embodiment. Next, an example of the operation of the gas image processing device 1 executed by a program as the gas image processing method will be described.

[0035] In step SA1(1), the first image processing unit 11 acquires a first time-series image MD1 from the first image acquisition unit 101. In step SA1(2), the first image processing unit 11 performs frequency processing on the first time-series image MD1 to extract gas signals, thereby acquiring a first frequency-processed image MD10. In step SA1(3), the first image processing unit 11 acquires a gas candidate region CL11 from the first frequency-processed image MD10.

[0036] In step SA2(1), the second image processing unit 12 acquires a second time-series image MD2 from the second image acquisition unit 102. In step SA2(2), the second image processing unit 12 performs frequency processing on the second time-series image MD2 to extract a gas exclusion signal, thereby acquiring a second frequency-processed image MD20. In step SA2(3), the second image processing unit 12 acquires a gas exclusion region CL21 from the second frequency-processed image MD20.

[0037] In step SA3, the calculation unit 13 compares the gas candidate region CL11 with the gas excluded region CL21 and removes noise. Then, in step SA4, the calculation unit 13 obtains the gas region Eg based on the gas candidate region CL11 and the gas excluded region CL21.

[0038] The operation will be described in detail below. Figures 3A, 3B, 3C, and 3D are explanatory diagrams showing an example of time-series infrared images. The first image acquisition unit 101 acquires a plurality of time-series infrared images MD1(1) to MD1(4). The infrared images MD1(1) to MD1(4) include the gas region to be monitored. The infrared image MD1(1) is the first infrared image at the start of imaging. The infrared image MD1(2) is the infrared image acquired 0.2 seconds later, the infrared image MD1(3) is the infrared image acquired 0.4 seconds later, and the infrared image MD1(4) is the infrared image acquired 0.6 seconds later. The number of infrared images acquired and the time intervals are examples.

[0039] The first image processing unit 11 performs image processing on these infrared images, such as that described in Japanese Patent No. 6245418. The image processing described in this publication involves frequency processing that captures changes in gas as it sways in the wind and extracts gas signals. This results in first frequency-processed images MD10(1) to MD10(4) from which the gas signals have been extracted.

[0040] 4A, 4B, 4C, and 4D are explanatory diagrams showing examples of first frequency-processed images from which gas signals have been extracted. The first frequency-processed image MD10(1) is an image from which gas signals have been extracted from the infrared image MD1(1). The first frequency-processed image MD10(2) is an image from which gas signals have been extracted from the infrared image MD1(2). The first frequency-processed image MD10(3) is an image from which gas signals have been extracted from the infrared image MD1(3). The first frequency-processed image MD10(4) is an image from which gas signals have been extracted from the infrared image MD1(4).

[0041] Next, the first image processing unit 11 acquires a gas candidate region CL11 from the first frequency-processed image MD10 from which the gas signal has been extracted. The first image processing unit 11 obtains the maximum value of the gas signal from the first frequency-processed image MD10 for n seconds from which the gas signal has been extracted, for example, from the first frequency-processed image MD10 for 1 second.

[0042] The first image processing unit 11 performs noise removal and threshold processing using known methods on the first frequency-processed image MD10(max) where the gas signal has the maximum value, thereby acquiring a gas candidate region CL11.

[0043] 5A is an explanatory diagram showing an example of a first frequency-processed image in which the gas signal has a maximum value, and FIG. 5B is an explanatory diagram showing an example of a gas candidate region. The method for acquiring the gas signal and the method for acquiring the gas candidate region are merely examples, and other methods may be used. When multiple gas candidate regions exist in one image, a labeling process is performed using a known method, and each gas candidate region is labeled.

[0044] 6A, 6B, 6C, and 6D are explanatory diagrams showing an example of time-series visible images. The second image acquisition unit 102 acquires a plurality of time-series visible images MD2(1) to MD2(4). The visible images MD2(1) to MD2(4) include the gas region to be monitored. The visible image MD2(1) is the first visible image at the start of imaging. The visible image MD2(2) is the visible image 0.2 seconds later, the visible image MD2(3) is the visible image 0.4 seconds later, and the visible image MD2(4) is the visible image 0.6 seconds later. The number of visible images acquired and the time intervals are merely examples.

[0045] The second image processing unit 12 performs graying processing on these visible images using a known method. The second image processing unit 12 performs image processing on the grayed visible images, similar to the infrared images, such as that described in Japanese Patent No. 6245418. The visible images are subjected to frequency processing that captures changes in gas swaying due to wind and extracts gas signals, thereby extracting gas exclusion signals such as steam swaying due to wind. The gas exclusion signals are noise components in the gas signals. This results in second frequency-processed images MD20(1) to MD20(4) from which the gas exclusion signals have been extracted.

[0046] 7A, 7B, 7C, and 7D are explanatory diagrams showing examples of second frequency-processed images from which gas exclusion signals have been extracted. The second frequency-processed image MD20(1) is an image from which the gas exclusion signals have been extracted from the visible image MD2(1). The second frequency-processed image MD20(2) is an image from which the gas exclusion signals have been extracted from the visible image MD2(2). The second frequency-processed image MD20(3) is an image from which the gas exclusion signals have been extracted from the visible image MD2(3). The second frequency-processed image MD20(4) is an image from which the gas exclusion signals have been extracted from the visible image MD2(4).

[0047] Next, the second image processing unit 12 acquires a gas exclusion region CL21 from the second frequency-processed image MD20 from which the gas exclusion signal has been extracted. The second image processing unit 12 obtains the maximum value of the gas exclusion signal from the second frequency-processed image MD20 for n seconds from which the gas exclusion signal has been extracted, for example, from the second frequency-processed image MD20 for 1 second.

[0048] The second image processing unit 12 performs noise removal and threshold processing using a known method on the second frequency-processed image MD20(max) where the gas exclusion signal has the maximum value, thereby acquiring a gas exclusion region CL21.

[0049] 8A is an explanatory diagram showing an example of a second frequency-processed image in which the gas exclusion signal has a maximum value, and FIG. 8B is an explanatory diagram showing an example of a gas exclusion region. The method for acquiring the gas exclusion signal and the method for acquiring the gas exclusion region are merely examples, and other methods may be used. When multiple gas exclusion regions exist in a single image, a labeling process is performed using a known method to label each gas exclusion region.

[0050] The calculation unit 13 uses the gas candidate region CL11 and the gas exclusion region CL21 to determine whether the region is a gas region or noise, and determines the gas region Eg. When multiple gas candidate regions exist within one image, the gas candidate regions divided by labeling are indicated as gas candidate regions CL11(h) (h = 1, 2 ... m). When multiple gas exclusion regions exist within one image, the gas exclusion regions divided by labeling are indicated as gas exclusion regions CL21(i) (i = 1, 2 ... n).

[0051] 9A and 9B are explanatory diagrams showing a first embodiment of a gas region determination method. Fig. 9A shows a case where multiple gas candidate regions exist in a single image, and the gas candidate regions indicated by CL11(1) to CL11(3) are gas candidate regions divided by labeling. Fig. 9B shows a gas exclusion region.

[0052] The calculation unit 13 compares the area ratios of the gas candidate regions CL11(1) to CL11(3) shown in FIG. 9A with the gas exclusion region CL21 shown in FIG. 9B. Specifically, the calculation unit 13 calculates the proportion of the area of ​​the gas candidate region CL11(1) that is overlapped by the gas candidate region CL11(1) and the gas exclusion region CL21 to the area of ​​the gas candidate region CL11(1). Similarly, the calculation unit 13 calculates the proportion of the area of ​​the gas candidate region CL11(2) that is overlapped by the gas exclusion region CL21 to the area of ​​the gas candidate region CL11(2). Furthermore, the calculation unit 13 calculates the proportion of the area of ​​the gas candidate region CL11(3) that is overlapped by the gas exclusion region CL21 to the area of ​​the gas candidate region CL11(3). The overlapping region of the gas candidate region CL11(1) and the gas exclusion region CL21 is considered to be noise other than gas. Similarly, the overlapping region between the gas candidate region CL11(2) and the gas exclusion region CL21 is considered to be noise other than gas. Furthermore, the overlapping region between the gas candidate region CL11(3) and the gas exclusion region CL21 is considered to be noise other than gas. Therefore, the calculation unit 13 determines that the overlapping region is noise if the ratio is equal to or greater than a predetermined threshold (e.g., 60%). If the ratio is less than the threshold, the calculation unit 13 determines that the overlapping region is a gas region and outputs this as the detection result for the gas region Eg.

[0053] 10A and 10B are explanatory diagrams showing a second embodiment of a gas region determination method. The comparison between the gas candidate region CL11 and the gas exclusion region CL21 may be performed using the distance between the centroid coordinates of the two regions. The calculation unit 13 compares the distance between the centroid coordinate O1 of the gas candidate region CL11 shown in FIG. 10A and the centroid coordinate O2 of the gas exclusion region CL21 shown in FIG. 10B. FIG. 10A shows a case where multiple gas candidate regions exist within a single image, and the gas candidate regions indicated by CL11(1) to CL11(3) are gas candidate regions divided by labeling. FIG. 10B shows a case where multiple gas exclusion regions exist within a single image, and the gas exclusion regions indicated by CL21(1) to CL21(2) are gas exclusion regions divided by labeling.

[0054] The calculation unit 13 calculates the barycentric coordinate O1(1) of the gas candidate region CL11(1) shown in FIG. 10A. The calculation unit 13 also calculates the barycentric coordinate O2(i) (i = 1, 2) of each gas exclusion region CL21(i) (i = 1, 2) shown in FIG. 10B. The calculation unit 13 then determines the distance between the gas candidate region CL11(1) and the center of gravity of each gas exclusion region CL21(i) (i = 1, 2). Similarly, the calculation unit 13 calculates the barycentric coordinate O1(2) of the gas candidate region CL11(2).

[0055] The calculation unit 13 also calculates the barycentric coordinates O2(i) (i = 1, 2) of each gas excluded region CL21(i) (i = 1, 2). Then, the calculation unit 13 calculates the distance between the gas candidate region CL11(2) and the center of gravity of each gas excluded region CL21(i) (i = 1, 2). Then, the calculation unit 13 calculates the barycentric coordinates O1(3) of the gas candidate region CL11(3). Then, the calculation unit 13 calculates the barycentric coordinates O2(i) (i = 1, 2) of each gas excluded region CL21(i) (i = 1, 2). Then, the calculation unit 13 calculates the distance between the gas candidate region CL11(3) and the center of gravity of each gas excluded region CL21(i) (i = 1, 2). Furthermore, it is determined whether the barycentric coordinate O1(1) of the gas candidate region CL11(1) is close to the barycentric coordinate O2(i) of each gas excluded region CL21(i). If there is at least one gas candidate region CL11(1) whose barycentric coordinate O1(1) is close to the barycentric coordinate O2(i), the gas candidate region CL11(1) is deemed to be noise other than gas. Similarly, it is determined whether the barycentric coordinate O1(2) of the gas candidate region CL11(2) is close to the barycentric coordinate O2(i) of each gas excluded region CL21(i). If there is at least one gas candidate region CL11(2) whose barycentric coordinate O1(2) is close to the barycentric coordinate O2(i), the gas candidate region CL11(2) is deemed to be noise. The calculation unit 13 also determines whether the centroid coordinate O1(3) of the gas candidate region CL11(3) is close to the centroid coordinate O2(i) of each gas exclusion region CL21(i). If there is at least one gas candidate region CL11(3) whose centroid coordinate O1(3) is close to the centroid coordinate O2(i), the calculation unit 13 determines the gas candidate region CL11(3) as noise. Therefore, if the centroid distance is less than a predetermined threshold (e.g., 10 pixels), the calculation unit 13 determines the gas candidate region as noise. Furthermore, if the centroid distances of all the gas candidate regions are equal to or greater than the predetermined threshold, the calculation unit 13 determines the gas region as a gas region and outputs this as the detection result of the gas region Eg.

[0056] In a third embodiment of the gas region determination method, the gas candidate region CL11 and the gas exclusion region CL21 may be compared based on the degree of shape similarity. The calculation unit 13 calculates the degree of match, for example, normalized cross-correlation (NCC) by pattern matching. The NCC can be calculated using the following formula, where I(x, y) is the pixel value of the input image, T(x, y) is the pixel value of the template image, w is the width of the template image, h is the height, and dx, dy are the scanning positions.

[0057]

[0058] When multiple gas candidate regions exist within a single image, the gas candidate regions indicated by CL11(h) are gas candidate regions divided by labeling. When multiple gas exclusion regions exist within a single image, the gas exclusion regions indicated by CL21(i) are gas exclusion regions divided by labeling. The third embodiment uses the example of h = 1, 2, 3 and i = 1, 2. A larger NCC value indicates a higher degree of similarity. Therefore, the calculation unit 13 calculates the NCC between each gas candidate region CL11(h) (h = 1, 2, 3) and each gas exclusion region CL21(i) (i = 1, 2). The calculation unit 13 determines that a gas candidate region CL11(h) having a gas exclusion region CL21(i) with an NCC equal to or greater than a predetermined threshold (e.g., 0.7) is noise. Furthermore, if all of the NCCs are less than the threshold value, the calculation unit 13 determines that the gas candidate region CL11(h) is a gas region, and outputs this as the detection result of the gas region Eg.

[0059] 11A and 11B are explanatory diagrams showing a fourth embodiment of the gas region determination method. The comparison between the gas candidate region CL11 and the gas exclusion region CL21 may be performed based on the similarity of their contours. FIG. 11A shows a case where multiple gas candidate regions exist within a single image, and the gas candidate regions indicated by CL11(1) to CL11(3) are gas candidate regions divided by labeling. FIG. 11B shows a case where multiple gas exclusion regions exist within a single image, and the gas exclusion regions indicated by CL21(1) to CL21(2) are gas exclusion regions divided by labeling.

[0060] The calculation unit 13 extracts a contour line L1(1) of the gas candidate region CL11(1) shown in FIG. 11A. The calculation unit 13 also extracts a contour line L1(2) of the gas candidate region CL11(2). The calculation unit 13 also extracts a contour line L1(3) of the gas candidate region CL11(3). The calculation unit 13 also extracts a contour line L2(1) of the gas excluded region CL21(1) and a contour line L2(2) of the gas excluded region CL21(2) shown in FIG. 11B. The contour lines of each region can be extracted using a known technique such as a Sobel filter or a Laplacian filter. The calculation unit 13 calculates HuMoments, which are represented by seven elements that are invariant to scale, position, and rotation, for each contour. The similarity of the contours can be calculated by calculating the distance between the HuMoments. HuMoments is defined as follows:

[0061]

[0062] nu ｊｉ is calculated using the following formula:

[0063]

[0064] Here, mu ｊｉ is the central moment, and m ｊｉ is the spatial moment. ｊｉ is expressed by the following formula:

[0065]

[0066] Also, m ｊｉ is expressed by the following formula:

[0067]

[0068] Then, when the contour of the gas candidate region CL11 is A and the contour of the gas excluded region CL21 is B, the HuMoments of each are expressed as h ｉ Ａ , h ｉ Ｂ If (i=1, 2, . . . , 7), the distance D of HuMoments can be calculated, for example, as follows:

[0069]

[0070] Here, m ｉ Ａ , m ｉ Ｂ is expressed by the following formula:

[0071]

[0072] The smaller the HuMoments distance D(A, B), the higher the similarity. Therefore, the calculation unit 13 calculates the HuMoments distance D(A, B) between each gas candidate region CL11(h) (h = 1, 2, 3) and each gas exclusion region CL21(i) (i = 1, 2). The calculation unit 13 determines a gas candidate region CL11(h) that has at least one gas exclusion region CL21(i) whose HuMoments distance D(A, B) is less than a predetermined threshold (e.g., 0.4) to be noise. The calculation unit 13 also determines a gas candidate region CL11(h) whose HuMoments distances D(A, B) are all equal to or greater than the threshold to be a gas region, and outputs this as the detection result for the gas region Eg.

[0073] 12A and 12B are explanatory diagrams showing a fifth embodiment of the gas region determination method. The gas candidate region CL11 and the gas exclusion region CL21 may be compared based on the similarity of the movement vectors between the two regions. The calculation unit 13 calculates the optical flow for the pixels in the gas candidate region CL11 shown in FIG. 10A and the pixels in the gas exclusion region CL21 shown in FIG. 10B. The optical flow is calculated using techniques such as the gradient method and the Lucas-Kanade method, which are well-known techniques.

[0074] Figure 12A shows a case where there are multiple gas candidate regions in a single image, and the gas candidate regions indicated by CL11(1) to (3) are gas candidate regions divided by labeling. Figure 12B shows a case where there are multiple gas exclusion regions in a single image, and the gas exclusion regions indicated by CL21(1) to (2) are gas exclusion regions divided by labeling.

[0075] The calculation unit 13 calculates the average value of the optical flow for pixels in the gas candidate regions CL11(1), CL11(2), and CL11(3). The calculation unit 13 also calculates the average value of the optical flow for pixels in the gas excluded regions CL21(1) and CL21(2).

[0076] The calculation unit 13 obtains the average value of the optical flow for pixels in the gas candidate region CL11(1) as the movement vector B1(1) for the gas candidate region CL11(1). The calculation unit 13 also obtains the average value of the optical flow for pixels in the gas candidate region CL11(2) as the movement vector B1(2) for the gas candidate region CL11(2). The calculation unit 13 also obtains the average value of the optical flow for pixels in the gas candidate region CL11(3) as the movement vector B1(3) for the gas candidate region CL11(3). The calculation unit 13 also obtains the average value of the optical flow for pixels in the gas excluded region CL21(1) as the movement vector B2(1) for the gas excluded region CL21(1). Furthermore, the calculation unit 13 obtains the average value of the optical flow at the pixels in the gas excluded region CL21(2) as the movement vector B2(2) of the gas excluded region CL21(2).

[0077] Gas candidate regions CL11(h) where the angles between the movement vectors B1(h) (h = 1, 2, 3) and B2(i) (i = 1, 2) are close are considered to be noise other than gas. Therefore, the calculation unit 13 calculates the angle between the movement vectors of each gas exclusion region CL21(i) and the gas candidate region CL11(h). The calculation unit 13 determines that a region is noise if one or more of the movement vectors have an angle less than a predetermined threshold (e.g., 30 degrees). Furthermore, the calculation unit 13 determines that a region is a gas region if all of the angles between the movement vectors are equal to or greater than the threshold, and outputs this as the detection result for the gas region Eg.

[0078] The method for calculating the motion vector is not limited to optical flow, and the motion vector may be calculated by template matching such as Sum of Absolute Difference (SAD) or Normalized Cross-Correlation (NCC).

[0079] Furthermore, the comparison between the gas candidate region CL11 and the gas excluded region CL21 may be performed by combining two or more of the above-described first to fifth embodiments.

[0080] The present invention can be used in a fluid image processing device, a fluid image processing system, a program, and a fluid image processing method that detect leakage of a fluid such as gas from time-series images.

[0081] REFERENCE SIGNS LIST 1 gas image processing device, 11 first image processing unit, 12 second image processing unit, 13 calculation unit, 100 gas image processing system, 101 first image acquisition unit, 102 second image acquisition unit, 200 information processing device, 201 control unit, 202 output unit

Claims

1. A fluid image processing apparatus comprising: a first image processing unit that performs frequency processing for extracting a monitoring target on a first time-series image captured with light of a first wavelength corresponding to a fluid to be monitored, and obtains a monitoring target candidate region; a second image processing unit that performs frequency processing for extracting a monitoring target on a second time-series image captured with light of a second wavelength different from the first wavelength, and obtains a monitoring target exclusion region; and an arithmetic unit that obtains a monitoring target based on the monitoring target candidate region and the monitoring target exclusion region.

2. The fluid image processing apparatus according to claim 1, wherein the arithmetic unit obtains a gas region by comparing a gas candidate region and a gas exclusion region.

3. The fluid image processing apparatus according to claim 1, wherein the first image processing unit is connected to a first image acquisition unit capable of capturing an infrared image.

4. The fluid image processing apparatus according to claim 1, wherein the second image processing unit is connected to a second image acquisition unit capable of capturing a visible image.

5. The fluid image processing apparatus according to claim 3, wherein the second image processing unit is connected to a second image acquisition unit capable of capturing an infrared image having a wavelength different from that of the first image acquisition unit.

6. The fluid image processing apparatus according to claim 1, wherein the arithmetic unit obtains a gas region by comparing the area ratio of the gas candidate region and the gas exclusion region.

7. The fluid image processing apparatus according to claim 1, wherein the arithmetic unit obtains a gas region by comparing the centroid coordinates of the gas candidate region and the gas exclusion region.

8. The fluid image processing apparatus according to claim 1, wherein the arithmetic unit obtains a gas region by comparing the shapes of the respective regions of the gas candidate region and the gas exclusion region.

9. The fluid image processing apparatus according to claim 1, wherein the arithmetic unit obtains a gas region by comparing the contours of the respective regions of the gas candidate region and the gas exclusion region.

10. The fluid image processing apparatus according to claim 1, wherein the arithmetic unit obtains a gas region by comparing the movement vectors of the gas candidate region and the gas exclusion region.

11. The fluid image processing apparatus according to claim 1, comprising a first image acquisition unit connected to the first image processing unit and capable of capturing an image with light of a first wavelength.

12. The fluid image processing apparatus according to claim 1, comprising a first image acquisition unit connected to the first image processing unit and capable of capturing an image with light of a first wavelength, and a second image acquisition unit connected to the second image processing unit and capable of capturing an image with light of a second wavelength different from the first wavelength.

13. A fluid image processing system comprising: a first image acquisition unit that acquires a first time-series image captured with light of a first wavelength corresponding to a fluid to be monitored; a second image acquisition unit that acquires a second time-series image captured with light of a second wavelength different from the first wavelength; and a fluid image processing device that performs image processing on the first time-series image and the second time-series image. The fluid image processing device includes: a first image processing unit that performs frequency processing for extracting a monitoring target on the first time-series image to obtain a monitoring target candidate region; a second image processing unit that performs frequency processing for extracting a monitoring target on the second time-series image to obtain a monitoring target exclusion region; and an arithmetic unit that obtains a monitoring target based on the monitoring target candidate region and the monitoring target exclusion region.

14. A program executed by a fluid image processing device that performs image processing on a time-series image, the program causing: a first image processing unit to perform frequency processing for extracting a monitoring target on a first time-series image captured with light of a first wavelength corresponding to a fluid to be monitored to obtain a monitoring target candidate region; a second image processing unit to perform frequency processing for extracting a monitoring target on a second time-series image captured with light of a second wavelength different from the first wavelength to obtain a monitoring target exclusion region; and an arithmetic unit to perform a step of obtaining a monitoring target based on the monitoring target candidate region and the monitoring target exclusion region.

15. A fluid image processing method including: performing frequency processing for extracting a monitoring target on a first time-series image captured with light of a first wavelength corresponding to a fluid to be monitored to obtain a monitoring target candidate region; performing frequency processing for extracting a monitoring target on a second time-series image captured with light of a second wavelength different from the first wavelength to obtain a monitoring target exclusion region; and performing a step of obtaining a monitoring target based on the monitoring target candidate region and the monitoring target exclusion region.

Images