Image processing device, image processing system, program, and image processing method

Abstract

The present invention provides an image processing device capable of reducing line noise from an image including gas.　An image processing device 1 comprises: a difference image generation unit 21 that generates a difference image from a reference image and a time-series input image that is input at a time different from the reference image; a line noise extraction unit 22 that extracts line noise in the longitudinal direction and / or the lateral direction from the difference image and generates a line noise image; and a corrected image generation unit 23 that generates a corrected image by using the line noise image.

Description

Image processing device, image processing system, program, and image processing method

[0001] The present disclosure relates to an image processing device, an image processing system, a program, and a fluid image processing method for detecting leakage of a fluid such as gas from time-series images.

[0002] 2. Description of the Related Art A technique has been proposed for detecting leakage of fluids such as gas using infrared images obtained by photographing a subject, including a monitoring target, with an infrared camera.

[0003] Temperature changes can cause vertical and horizontal line noise (called line noise) to appear in different positions in each frame on the image sensor. Gas only changes slightly (appears faintly) on the infrared image, so high sensitivity is required to visualize it. However, even the slightest line noise can make the gas difficult to see.

[0004] In response to this, a method is known in which a recursive filter is used to reduce noise over time (see, for example, Patent Document 1).

[0005] JP 2011-134118 A

[0006] A method for reducing noise over time by extracting areas of potential line noise from an input image and applying a recursive filter reduces line noise in the input image. However, because gas is only visible very faintly in this method, the changes caused by the gas are smoothed over time along with the noise, causing the gas to disappear. For this reason, this method could not be applied to reduce line noise from images containing gas.

[0007] Furthermore, methods that remove line noise from a single input image at any given time leave line noise with minute changes comparable to the random noise of the image sensor. When visualizing gas using infrared images, it is difficult to see the gas in the infrared image alone, so the gas is visualized with high sensitivity using difference images that extract changes between images in a time series. However, in difference images that extract changes between images in a time series, line noise with minute changes comparable to random noise stands out, making gas visibility worse.

[0008] The present disclosure has been made to solve such problems, and aims to provide an image processing device that can reduce line noise from images containing gas. The present disclosure also aims to provide an image processing system including the image processing device, a program executed by the image processing device, and an image processing method.

[0009] In order to solve the above-mentioned problems, the present disclosure provides an image processing device including a differential image generation unit that generates a differential image from a reference image and an input image of a time series having a different input timing from the reference image, a line noise extraction unit that extracts vertical and / or horizontal line noise from the differential image and generates a vertical or horizontal line noise image, or a vertical line noise image and a horizontal line noise image, and a corrected image generation unit that generates a corrected image using the line noise image.

[0010] The present disclosure also provides an image processing system including an image processing device, the image processing device including a differential image generation unit that generates a differential image from a reference image and an input image in a time series having a different input timing from the reference image, a line noise extraction unit that extracts vertical and / or horizontal line noise from the differential image and generates a vertical or horizontal line noise image, or a vertical line noise image and a horizontal line noise image, and a corrected image generation unit that generates a corrected image using the line noise image.

[0011] Furthermore, the present disclosure is a program executed by an image processing unit that performs image processing on time-series images, causing the image processing unit to execute the steps of generating a difference image from a reference image and a time-series input image having an input timing different from that of the reference image, extracting vertical and / or horizontal line noise from the difference image to generate a vertical or horizontal line noise image, or a vertical line noise image and a horizontal line noise image, and generating a corrected image using the line noise image.

[0012] The present disclosure also provides an image processing method that executes the steps of generating a difference image from a reference image and an input image of a time series having a different input timing from the reference image, extracting vertical and / or horizontal line noise from the difference image to generate a vertical or horizontal line noise image, or a vertical line noise image and a horizontal line noise image, and generating a corrected image using the line noise image.

[0013] In the present disclosure, vertical and / or horizontal line noise is extracted from a difference image between a reference image and an input image, from which changes between images in a time series are extracted, thereby generating a line noise image from which random noise components have been removed.

[0014] Therefore, the visibility of gas is improved in frequency-processed images obtained by performing frequency processing to extract gas signals on time-series corrected images.

[0015] FIG. 1 is a functional block diagram showing an example of an image processing device and an image processing system according to the present embodiment; FIG. 2 is a flowchart showing an example of an image processing method according to the present embodiment; FIG. 3 is an image transition diagram showing an example of the image processing method according to the present embodiment; FIG. 4 is an image transition diagram showing another example of the image processing method according to the present embodiment; FIG. 5 is an explanatory diagram showing an example of time-series infrared images; FIG. 6 is an explanatory diagram showing an example of a difference signal extracted image for corrected images of time-series input images; FIG. 7 is an explanatory diagram showing an example of a frequency-processed image in which a gas signal has been extracted for corrected images of time-series input images; FIG. 8 is an explanatory diagram showing an example of a difference signal extracted image for uncorrected time-series input images; FIG. 9 is an explanatory diagram showing an example of a frequency-processed image in which a gas signal has been extracted for uncorrected time-series input images; FIG. 10 is an image transition diagram showing an example of an image processing method corresponding to a moving object according to the present embodiment; FIG. 11 is an image transition diagram showing another example of an image processing method corresponding to a moving object according to the present embodiment; FIG. 12 is an explanatory diagram showing an example of a difference signal extracted image for corrected images corresponding to a moving object in time-series input images; FIG. 13 is an explanatory diagram showing an example of a frequency-processed image in which a gas signal has been extracted for corrected images corresponding to a moving object in time-series input images; FIG. 14 is an explanatory diagram showing an example of a difference signal extracted image for corrected images not corresponding to a moving object in time-series input images. FIG. 10 is an explanatory diagram showing an example of a frequency-processed image in which a gas signal is extracted from a corrected image that does not correspond to a moving object in a time-series input image.

[0016] Hereinafter, embodiments of an image processing device, an image processing system, a program, and an image processing method according to the present disclosure will be described with reference to the drawings.

[0017] <Configuration Example of Image Processing Apparatus and Image Processing System> FIG. 1 is a functional block diagram showing an example of an image processing apparatus and an image processing system according to the present embodiment.

[0018] The image processing device 1 includes an image acquisition unit 10, an image processing unit 20, and an image storage unit 30. The image processing system 100 includes the image processing device 1 and an output unit 40.

[0019] The image acquisition unit 10 is a camera equipped with an optical system, a filter, an image sensor, a signal processing unit, etc. (not shown). The image acquisition unit 10 is configured to pass only light in a wavelength range corresponding to the monitored object, for example, through a filter, and receive the light with the image sensor. The monitored object is gas. Gas is an example of a leaking fluid.

[0020] The wavelength range corresponding to the type of gas to be photographed is, for example, infrared. A thermography camera with a wavelength range of far infrared (approximately 7 to 14 μm) can capture infrared images of ammonia and ethylene. The camera used for photographing is not limited to a thermography camera; a near-infrared camera or mid-infrared camera with a wavelength range selected according to the type of gas to be photographed can also be used.

[0021] The image acquisition unit 10 captures time-series still images, video images, etc. of infrared images of a subject including a gas leak monitoring target, and acquires time-series images. The image acquisition unit 10 may be configured to be included in the image processing device 1, or may be configured as a camera independent of the image processing device 1 and connected to the image processing device 1.

[0022] The image processing unit 20 includes a differential image generating unit 21, a line noise extracting unit 22, a corrected image generating unit 23, and a gas extracting unit 24. The image processing unit 20 is an example of a control unit, and is configured by a CPU (Central Processing Unit) or the like.

[0023] The difference image generation unit 21 creates a difference image from a reference image and an input image in a time series that has a different input timing from the reference image. The reference image is, for example, the first frame image of the time series. If the reference image is the first frame image of the time series, the input image is a frame image of the same time series that has a different timing from the first frame image. The difference image generation unit 21 subtracts the reference image from the input image to generate a difference image. The reference image does not have to be the first frame image, and can be any frame image, such as the image one frame before the input image.

[0024] The line noise extraction unit 22 extracts line noise from the difference image and generates a vertical and / or horizontal line noise image. The line noise extraction unit 22 takes the vertical average value of the difference image and generates a line noise image along the vertical direction where the average value was taken. The line noise extraction unit 22 also takes the horizontal average value of the difference image and generates a line noise image along the horizontal direction where the average value was taken.

[0025] The corrected image generation unit 23 generates a corrected image in which line noise has been corrected using the line noise image. The corrected image generation unit 23 generates a corrected image in which line noise has been corrected for the input image using the vertical and / or horizontal line noise image. The corrected image generation unit 23 may also generate a corrected image in which line noise has been corrected for the difference image using the vertical and / or horizontal line noise image.

[0026] The gas extraction unit 24 performs frequency processing for extracting gas from the time-series corrected images, and extracts gas signals.

[0027] The image storage unit 30 stores the time-series images acquired by the image acquisition unit 10. The image storage unit 30 also stores a reference image selected from the time-series images.

[0028] The output unit 40 is configured with a display or the like. The output unit 40 is, for example, a liquid crystal display. Instead of a liquid crystal display, the output unit 40 may be an organic light emitting diode (EL) display, a plasma display, or the like. The output unit 40 may be provided in the image processing device 1. The output unit 40 may be a display of a mobile terminal device connected to the image processing device 1, or a display of a computer connected to the image processing device 1.

[0029] <Example of Program Functions> The image processing unit 20 executes a program to realize the functions of the difference image generating unit 21, the line noise extracting unit 22, the corrected image generating unit 23, and the gas extracting unit 24 described above.

[0030] The program causes the difference image generating unit 21 to execute a difference process for generating a difference image between a reference image and an input image.

[0031] The program also causes a line noise extraction unit 22 to execute a line noise extraction process for extracting line noise from the differential image and generating a line noise image.

[0032] Furthermore, the program causes the corrected image generating unit 23 to execute a line noise correction process for generating a corrected image in which line noise has been corrected using the line noise image.

[0033] The program also causes the gas extracting unit 24 to execute a gas image extraction process for extracting gas signals.

[0034] Furthermore, the program causes the steps of differential processing, line noise extraction processing, line noise correction processing, and gas image extraction processing to be executed in a predetermined order.

[0035] The program is stored in RAM (Random Access Memory), ROM (Read Only Memory), or the like. In a configuration in which the image processing device 1 includes a hard disk drive (HDD), a solid state drive (SSD), or the like, the program may be stored in the HDD, SSD, or the like. Furthermore, in a configuration in which the image processing device 1 is connectable to an HDD, SSD, or the like as an external storage medium, the program may be stored in the external storage medium and provided. Furthermore, the program may be stored in a server connected to the image processing device 1 via a wired or wireless network, and the image processing device 1 may receive the program via the network.

[0036] Furthermore, by updating the program, it may be possible to add functions such as differential processing, line noise extraction processing, and line noise correction processing.

[0037] In the image processing unit 20, some or all of the differential image generating unit 21, line noise extracting unit 22, corrected image generating unit 23, and gas extracting unit 24 may be realized by processing by a CPU. Also, in the image processing unit 20, some or all of the above functions may be realized by processing by a DSP (Digital Signal Processor) instead of or in addition to a CPU. Furthermore, some or all of the above functions may be realized by processing by a dedicated hardware circuit instead of or in addition to software processing.

[0038] <Example of operation of image processing method> Fig. 2 is a flowchart showing an example of an image processing method according to this embodiment, and Fig. 3 is an image transition diagram showing an example of the image processing method according to this embodiment. Next, an example of the operation of the image processing device 1 executed by a program as the image processing method will be described.

[0039] The difference image generating unit 21 generates the reference image MD in the difference processing of step SA1. ０ and the same time series image, the reference image MD ０ Input image MD at subsequent input timings ｉ The differential image generating unit 21 obtains the input image MD ｉ From the reference image MD ０ and subtracting the difference image MD ｄｉｆ get.

[0040] In the line noise extraction process of step SA2, the line noise extraction unit 22 extracts the differential image MD ｄｉｆ Line noise is extracted from the image to generate a line noise image MD Ｌｎ Generate.

[0041] When performing correction using vertical line noise, the line noise extraction unit 22 extracts the difference image MD ｄｉｆ For each column L of the image, the line noise extracting unit 22 calculates the average value in the column direction and extracts the line noise. ｄｉｆ Line noise is extracted from the vertical line noise image MD Ｌｎ Generate.

[0042] In the line noise correction process of step SA3, the corrected image generating unit 23 generates a line noise image MD Ｌｎ Corrected image MD in which line noise is corrected using ｃ Generate.

[0043] Input image MD ｉ When correcting the vertical line noise of the image MD Ｌｎ The input image MD ｉ As a result, the corrected image generating unit 23 subtracts the input image MD ｉ Corrected image MD after correcting vertical line noise ｃ Generate.

[0044] Although not shown, the input image may be corrected using horizontal line noise. In this case, the line noise extraction unit 22 calculates the average value in the row direction for each row of the difference image to extract the line noise. As a result, the line noise extraction unit 22 extracts the line noise from the difference image and generates a horizontal line noise image.

[0045] The corrected image generating unit 23 subtracts the horizontal line noise image from the input image, thereby generating a corrected image in which the horizontal line noise of the input image has been corrected.

[0046] Furthermore, although not shown, the input image may be corrected using vertical and horizontal line noise. In this case, the line noise extraction unit 22 extracts column-direction line noise from the difference image to generate a vertical line noise image. The line noise extraction unit 22 also extracts row-direction line noise from the same difference image from which the vertical line noise image was obtained to generate a horizontal line noise image.

[0047] The corrected image generating unit 23 subtracts the vertical line noise image from the input image, and also subtracts the horizontal line noise image from the corrected input image in which the vertical line noise has been corrected.

[0048] Note that the vertical and horizontal line noise corrections can be performed in any order. That is, the corrected image generation unit 23 subtracts the horizontal line noise image from the input image. The corrected image generation unit 23 also subtracts the vertical line noise image from the corrected input image in which the horizontal line noise has been corrected. In this way, the corrected image generation unit 23 generates a corrected image in which the vertical and horizontal line noise of the input image have been corrected.

[0049] 4 is an image transition diagram showing another example of the image processing method according to the present embodiment. The difference image may be corrected using vertical and / or horizontal line noise. When correcting the difference image using vertical line noise, the corrected image generating unit 23 generates a vertical line noise image MD Ｌｎ , the difference image MD ｄｉｆ The difference image MD is subtracted from ｄｉｆ is this line noise image MD Ｌｎ As a result, the corrected image generating unit 23 generates the difference image MD ｄｉｆ Corrected image MD after correcting vertical line noise ｃ Generate.

[0050] When correcting the difference image using horizontal line noise, the corrected image generation unit 23 subtracts the horizontal line noise image from the difference image that generated this line noise image, thereby generating a corrected image in which the horizontal line noise of the difference image has been corrected.

[0051] When correcting the difference image using vertical and horizontal line noise, the corrected image generation unit 23 subtracts the vertical line noise image from the difference image generated by this line noise image, and also subtracts the horizontal line noise image from the difference image generated by this line noise image.

[0052] The corrected image generating unit 23 subtracts the vertical line noise image from the difference image, and also subtracts the horizontal line noise image from the difference image after correcting the vertical line noise.

[0053] Alternatively, the corrected image generation unit 23 subtracts the horizontal line noise image from the difference image. Also, the corrected image generation unit 23 subtracts the vertical line noise image from the corrected difference image in which the horizontal line noise has been corrected. In this way, the corrected image generation unit 23 generates a corrected image in which the vertical and horizontal line noise of the difference image have been corrected.

[0054] The gas extractor 24 extracts gas signals from the corrected image in gas image extraction processing in step SA4.

[0055] The gas extraction unit 24 performs image processing on the time-series corrected images, for example, as described in Japanese Patent No. 6245418 filed by the same applicant as the present invention. The image processing described in this publication involves frequency processing that captures changes in gas fluctuations caused by wind and extracts gas signals. This results in a frequency-processed image from which the gas signals have been extracted.

[0056] The operation will be described in detail below. FIG. 5 is an explanatory diagram showing an example of time-series infrared images. The image acquisition unit 10 acquires a plurality of time-series infrared images MD1(1) to MD1(4). The infrared images MD1(1) to MD1(4) include the gas region to be monitored. The infrared image MD1(1) is the first infrared image at the start of imaging. The infrared image MD1(2) is the infrared image acquired 0.2 seconds later, the infrared image MD1(3) is the infrared image acquired 0.4 seconds later, and the infrared image MD1(4) is the infrared image acquired 0.6 seconds later. The number of infrared images acquired and the time intervals are examples.

[0057] 6 is an explanatory diagram showing an example of difference signal extraction images for corrected images of time-series input images. Difference signal extraction image MD2(1) is an image in which a difference signal is extracted from a corrected image of an input image corresponding to infrared image MD1(1). Difference signal extraction image MD2(2) is an image in which a difference signal is extracted from a corrected image of an input image corresponding to infrared image MD1(2). Difference signal extraction image MD2(3) is an image in which a difference signal is extracted from a corrected image of an input image corresponding to infrared image MD1(3). Difference signal extraction image MD2(4) is an image in which a difference signal is extracted from a corrected image of an input image corresponding to infrared image MD1(4).

[0058] 7 is an explanatory diagram showing an example of frequency-processed images in which gas signals have been extracted from corrected images of time-series input images. Frequency-processed image MD3(1) is an image in which gas signals have been extracted from the corrected image of the input image corresponding to infrared image MD1(1). Frequency-processed image MD3(2) is an image in which gas signals have been extracted from the corrected image of the input image corresponding to infrared image MD1(2). Frequency-processed image MD3(3) is an image in which gas signals have been extracted from the corrected image of the input image corresponding to infrared image MD1(3). Frequency-processed image MD3(4) is an image in which gas signals have been extracted from the corrected image of the input image corresponding to infrared image MD1(4).

[0059] 8 is an explanatory diagram showing an example of difference signal extraction images for time-series uncorrected input images as a comparative example. Difference signal extraction image MD4(1) is an image in which a difference signal is extracted from an uncorrected input image corresponding to infrared image MD1(1). Difference signal extraction image MD4(2) is an image in which a difference signal is extracted from an uncorrected input image corresponding to infrared image MD1(2). Difference signal extraction image MD4(3) is an image in which a difference signal is extracted from an uncorrected input image corresponding to infrared image MD1(3). Difference signal extraction image MD4(4) is an image in which a difference signal is extracted from an uncorrected input image corresponding to infrared image MD1(4).

[0060] 9 is an explanatory diagram showing an example of frequency-processed images in which gas signals have been extracted from time-series uncorrected input images as a comparative example. Frequency-processed image MD5(1) is an image in which gas signals have been extracted from the uncorrected input image corresponding to infrared image MD1(1). Frequency-processed image MD5(2) is an image in which gas signals have been extracted from the uncorrected input image corresponding to infrared image MD1(2). Frequency-processed image MD5(3) is an image in which gas signals have been extracted from the uncorrected input image corresponding to infrared image MD1(3). Frequency-processed image MD5(4) is an image in which gas signals have been extracted from the uncorrected input image corresponding to infrared image MD1(4).

[0061] It can be seen that the line noise is reduced in the differential signal extraction image for the corrected image compared to the uncorrected differential signal extraction image. Also, it can be seen that the line noise is reduced in the frequency processed image in which the gas signal is extracted from the differential signal extraction image for the corrected image compared to the uncorrected frequency processed image. This shows that the visibility of gas is improved in the differential signal extraction image and frequency processed image for the corrected image.

[0062] In addition, the reference image MD ０ is the first frame image of the time series (T=t ０ In the above embodiment, the processing is continued without changing the reference image MD. ０ Also, the reference image MD may be reset for each frame (every time). ０ Furthermore, when changing the photographing range, the reference image MD ０ Reset the settings.

[0063] 10 is an image transition diagram showing an example of an image processing method for a moving object according to the present embodiment. Next, an example of an operation of the image processing device 1 executed by a program for a moving object will be described as an image processing method.

[0064] The difference image generating unit 21 generates the reference image MD in the difference processing of step SA1 described above. ０ and the same time series image, the reference image MD ０ Input image MD at subsequent input timings ｉ The differential image generating unit 21 obtains the input image MD ｉ From the reference image MD ０ and subtracting the difference image MD ｄｉｆ get.

[0065] The line noise extraction unit 22 extracts the differential image MD in the line noise extraction process of step SA2. ｄｉｆ Line noise is extracted from the image to generate a line noise image MD Ｌｎ Generate.

[0066] The line noise extraction unit 22 sets a threshold value for calculating the average value in the vertical and horizontal directions. When performing correction using vertical line noise, the line noise extraction unit 22 extracts the difference image MD ｄｉｆ For each column L of the difference image MD, the line noise extraction unit 22 calculates the average value in the column direction. In this case, if the difference value is greater than a threshold, the line noise extraction unit 22 excludes the difference value from the calculation of the average value, and only if the difference value is equal to or less than the threshold, the line noise extraction unit 22 includes the difference value in the calculation of the average value. Then, the line noise extraction unit 22 calculates the average value in the column direction using pixels whose difference value is equal to or less than the threshold, and extracts line noise. In this way, the line noise extraction unit 22 extracts the difference image MD ｄｉｆ Line noise is extracted from the vertical line noise image MD Ｌｎ Generate.

[0067] The corrected image generating unit 23 generates the line noise image MD in the line noise correction process of step SA3 described above. Ｌｎ Corrected image MD in which line noise is corrected using ｃ Generate.

[0068] Input image MD ｉ When correcting the vertical line noise of the image MD Ｌｎ The input image MD ｉ As a result, the corrected image generating unit 23 subtracts the input image MD ｉ Corrected image MD after correcting vertical line noise ｃ Generate.

[0069] Furthermore, even in cases corresponding to a moving object, the input image may be corrected using horizontal line noise (not shown). In this case, the line noise extraction unit 22 calculates the row average value for each row of the difference image. In this case, the line noise extraction unit 22 excludes difference values ​​greater than a threshold from the calculation of the average value, and includes only difference values ​​equal to or less than the threshold. The line noise extraction unit 22 then calculates the row average value using pixels whose difference values ​​are equal to or less than the threshold, thereby extracting line noise. In this way, the line noise extraction unit 22 extracts line noise from the difference image and generates a horizontal line noise image.

[0070] The corrected image generating unit 23 subtracts the horizontal line noise image from the input image, thereby generating a corrected image in which the horizontal line noise of the input image has been corrected.

[0071] Furthermore, even when dealing with a moving object, the input image may be corrected using vertical and horizontal line noise (not shown). In this case, the line noise extraction unit 22 calculates the average value in the column direction where the difference value is equal to or less than a threshold. Then, the line noise extraction unit 22 extracts the line noise in the column direction from the difference image to generate a vertical line noise image. The line noise extraction unit 22 also calculates the average value in the row direction where the difference value is equal to or less than a threshold. Then, the line noise extraction unit 22 extracts the line noise in the row direction from the same difference image from which the vertical line noise image was obtained to generate a horizontal line noise image.

[0072] The corrected image generating unit 23 subtracts the vertical line noise image from the input image, and also subtracts the horizontal line noise image from the corrected input image in which the vertical line noise has been corrected.

[0073] Note that the vertical and horizontal line noise corrections can be performed in any order. That is, the corrected image generation unit 23 subtracts the horizontal line noise image from the input image. The corrected image generation unit 23 also subtracts the vertical line noise image from the corrected input image in which the horizontal line noise has been corrected. In this way, the corrected image generation unit 23 generates a corrected image in which the vertical and horizontal line noise of the input image have been corrected.

[0074] 11 is an image transition diagram showing another example of the image processing method for a moving object according to the present embodiment. When processing a moving object, the difference image may be corrected using vertical and / or horizontal line noise.

[0075] When correcting the difference image using vertical line noise, the line noise extraction unit 22 calculates the average value in the column direction where the difference value is equal to or less than a threshold value. Then, the line noise extraction unit 22 extracts the line noise in the column direction from the difference image and generates a vertical line noise image MD ＬｎThe corrected image generating unit 23 generates the vertical line noise image MD Ｌｎ , the difference image MD ｄｉｆ The difference image MD is subtracted from ｄｉｆ is this line noise image MD Ｌｎ As a result, the corrected image generating unit 23 generates the difference image MD ｄｉｆ Corrected image MD after correcting vertical line noise ｃ Generate.

[0076] When correcting the difference image using horizontal line noise, the line noise extraction unit 22 calculates the average value in the row direction where the difference value is equal to or less than a threshold.The line noise extraction unit 22 then extracts the line noise in the row direction from the difference image to generate a horizontal line noise image.The corrected image generation unit 23 subtracts the horizontal line noise image from the difference image from which the line noise image was generated.In this way, the corrected image generation unit 23 generates a corrected image in which the horizontal line noise of the difference image has been corrected.

[0077] When correcting the difference image using vertical and horizontal line noise, the corrected image generation unit 23 subtracts the vertical line noise image from the difference image generated by this line noise image, and also subtracts the horizontal line noise image from the difference image generated by this line noise image.

[0078] The corrected image generating unit 23 subtracts the vertical line noise image from the difference image, and also subtracts the horizontal line noise image from the difference image after correcting the vertical line noise.

[0079] Alternatively, the corrected image generation unit 23 subtracts the horizontal line noise image from the difference image. Also, the corrected image generation unit 23 subtracts the vertical line noise image from the corrected difference image in which the horizontal line noise has been corrected. In this way, the corrected image generation unit 23 generates a corrected image in which the vertical and horizontal line noise of the difference image have been corrected.

[0080] In the gas image extraction process of step SA4 described above, the gas extraction unit 24 performs image processing such as that described in Japanese Patent No. 6245418, and extracts gas signals from the corrected image.

[0081] The details of the operation corresponding to a moving object are described below. FIG. 12 is an explanatory diagram showing an example of a difference signal extraction image for a corrected image corresponding to a moving object in a time-series input image. The difference signal extraction image MD20(1) is an image in which a difference signal is extracted from a corrected image corresponding to a moving object in an input image corresponding to the infrared image MD1(1) shown in FIG. The difference signal extraction image MD20(2) is an image in which a difference signal is extracted from a corrected image corresponding to a moving object in an input image corresponding to the infrared image MD1(2). The difference signal extraction image MD20(3) is an image in which a difference signal is extracted from a corrected image corresponding to a moving object in an input image corresponding to the infrared image MD1(3). The difference signal extraction image MD20(4) is an image in which a difference signal is extracted from a corrected image corresponding to a moving object in an input image corresponding to the infrared image MD1(4).

[0082] FIG. 13 is an explanatory diagram showing an example of frequency-processed images in which gas signals have been extracted from corrected images corresponding to moving objects in time-series input images. Frequency-processed image MD30(1) is an image in which gas signals have been extracted from a corrected image corresponding to a moving object in an input image corresponding to infrared image MD1(1). Frequency-processed image MD30(2) is an image in which gas signals have been extracted from a corrected image corresponding to a moving object in an input image corresponding to infrared image MD1(2). Frequency-processed image MD30(3) is an image in which gas signals have been extracted from a corrected image corresponding to a moving object in an input image corresponding to infrared image MD1(3). Frequency-processed image MD30(4) is an image in which gas signals have been extracted from a corrected image corresponding to a moving object in an input image corresponding to infrared image MD1(4).

[0083] FIG. 14 is an explanatory diagram showing, as a comparative example, an example of a difference signal extraction image for a corrected image that does not correspond to a moving object in a time-series input image. The difference signal extraction image MD40(1) is an image in which a difference signal is extracted from a corrected image that does not correspond to a moving object in an input image corresponding to infrared image MD1(1). The difference signal extraction image MD40(2) is an image in which a difference signal is extracted from a corrected image that does not correspond to a moving object in an input image corresponding to infrared image MD1(2). The difference signal extraction image MD40(3) is an image in which a difference signal is extracted from a corrected image that does not correspond to a moving object in an input image corresponding to infrared image MD1(3). The difference signal extraction image MD40(4) is an image in which a difference signal is extracted from a corrected image that does not correspond to a moving object in an input image corresponding to infrared image MD1(4).

[0084] FIG. 15 is an explanatory diagram showing, as a comparative example, an example of frequency-processed images in which gas signals have been extracted from corrected images that do not correspond to moving objects in time-series input images. Frequency-processed image MD50(1) is an image in which gas signals have been extracted from a corrected image that does not correspond to moving objects in the input image corresponding to infrared image MD1(1). Frequency-processed image MD50(2) is an image in which gas signals have been extracted from a corrected image that does not correspond to moving objects in the input image corresponding to infrared image MD1(2). Frequency-processed image MD50(3) is an image in which gas signals have been extracted from a corrected image that does not correspond to moving objects in the input image corresponding to infrared image MD1(3). Frequency-processed image MD50(4) is an image in which gas signals have been extracted from a corrected image that does not correspond to moving objects in the input image corresponding to infrared image MD1(4).

[0085] When an image contains moving objects such as people or cars, the difference signal extraction image for the corrected image that does not correspond to the moving object contains moving object components other than line noise components. Also, the frequency processed image in which the gas signal is extracted from the corrected image that does not correspond to the moving object contains moving object components other than line noise components. This is because when the average values ​​in the vertical and horizontal directions are calculated to extract the line noise, moving object components other than the line noise components are also extracted.

[0086] Therefore, in order to extract only the line noise components with minute changes, a threshold is set when calculating the average values ​​in the vertical and horizontal directions. If the difference value is greater than the threshold, it is excluded from the calculation of the average value, and only if the difference value is equal to or less than the threshold, it is included in the calculation of the average value.

[0087] As described above, it can be seen that the difference signal extraction image for the corrected image corrected for a moving object has reduced line noise without being affected by the moving object. Also, it can be seen that the frequency processed image in which a gas signal is extracted for the corrected image corrected for a moving object has reduced line noise without being affected by the moving object. This shows that the difference signal extraction image and frequency processed image for the corrected image corrected for a moving object have improved gas visibility.

[0088] The present disclosure can be used in an image processing device, an image processing system, a program, and an image processing method that are capable of reducing line noise from an image that includes gas.

[0089] 1... image processing device, 10... image acquisition unit, 20... image processing unit, 21... difference image generation unit, 22... line noise extraction unit, 23... corrected image generation unit, 24... gas extraction unit, 30... image storage unit, 40... display unit, 100... image processing system

Claims

1. An image processing device comprising: a differential image generation unit that generates a differential image from a reference image and an input image of a time series having a different input timing from the reference image; a line noise extraction unit that extracts vertical and / or horizontal line noise from the differential image and generates a vertical or horizontal line noise image, or a vertical line noise image and a horizontal line noise image; and a corrected image generation unit that generates a corrected image using the line noise image.

2. The image processing device according to claim 1, wherein the corrected image generating section corrects the input image using the line noise image to generate a corrected image.

3. The image processing device according to claim 1, wherein the corrected image generating section corrects the difference image using the line noise image to generate a corrected image.

4. The image processing device according to claim 1, wherein the corrected image generating unit generates the corrected image using a vertical line noise image.

5. The image processing device according to claim 1, wherein the corrected image generating section generates the corrected image using a horizontal line noise image.

6. The image processing device according to claim 1, wherein the corrected image generation unit generates a corrected image using a line noise image in either the vertical or horizontal direction, and generates a corrected image from the corrected image using a line noise image in the other vertical or horizontal direction.

7. The image processing device according to claim 1, wherein the line noise extraction unit calculates an average value in either the vertical or horizontal direction for each column or row of the difference image, and generates a line noise image along the direction in which the average value was calculated.

8. The image processing device according to claim 1, wherein the line noise extraction unit takes an average value for each column or row of the difference image using pixels whose difference value is equal to or less than a threshold value, and creates the line noise image.

9. The image processing device according to claim 1, further comprising a gas extraction unit that performs frequency processing to extract gas from the time-series corrected images.

10. An image processing system having an image processing device, the image processing device comprising: a differential image generation unit that generates a differential image from a reference image and an input image of a time series having an input timing different from that of the reference image; a line noise extraction unit that extracts vertical and / or horizontal line noise from the differential image and generates a vertical or horizontal line noise image, or a vertical line noise image and a horizontal line noise image; and a corrected image generation unit that generates a corrected image using the line noise image.

11. A program executed by an image processing unit that performs image processing on time-series images, causing the image processing unit to execute the following steps: generating a difference image from a reference image and a time-series input image that has a different input timing from the reference image; extracting vertical and / or horizontal line noise from the difference image to generate a vertical or horizontal line noise image, or a vertical line noise image and a horizontal line noise image; and generating a corrected image using the line noise image.

12. An image processing method comprising the steps of: generating a difference image from a reference image and an input image of a time series having a different input timing from the reference image; extracting vertical and / or horizontal line noise from the difference image to generate a vertical or horizontal line noise image, or a vertical line noise image and a horizontal line noise image; and generating a corrected image using the line noise image.

Images