Image processing device, image processing method and program

Abstract

To reduce the stains of an observation window in an imaging device reflected in a captured image, thus to obtain a more sharp in-furnace image.SOLUTION: An image processing device for subjecting an image obtained by imaging an in-furnace part to image processing comprises: a time representative value image creation part in which, from plural images obtained by successively imaging a furnace wall with an imaging device moving through the in-furnace part, time representative value images consisting of the representative value of the pixel values of the same pixel position is created; a filter creation part in which, based on each pixel value of the time representative value images and the pixel value of the pixel position at which the probability that disturbance upon imaging by the imaging device in the time representative value images is not generated is the highest, a filter arrangement for correcting the change of the pixel value caused by the disturbance upon the imaging by the imaging device is created; and a correction processing part for correcting the image imaged by the imaging device using the filter arrangement.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program for processing an image captured inside a coke oven. [Background technology]

[0002] In high-temperature furnace chambers such as the carbonization chamber of a coke oven, the furnace walls are made of refractory materials, and it is important for the operation management of furnaces such as coke ovens to accurately grasp the deterioration status of refractory materials. In particular, the carbonization chamber of a coke oven is operated continuously for a long period of time, usually more than 30 years, under severe conditions. The refractory bricks that make up the carbonization chamber gradually deteriorate due to thermal, chemical, and mechanical factors. This causes damage to the oven walls and clogging of coke due to poor carbon adhesion, which leads to further damage to the oven walls, delays in production schedules, and increased workload for recovery work. Therefore, it is extremely important for the operation management of coke ovens to constantly grasp the deterioration status of the refractory bricks in the carbonization chamber, especially the oven walls.

[0003] As a method for observing the walls of the coke oven carbonization chamber, for example, Patent Document 1 discloses a method for observing the inner walls of a coke oven carbonization chamber, in which a camera is inserted into the chamber through the mouth of the coke oven and the condition of the oven wall is observed based on images of the inner wall, the camera's field of view is set at an angle and the oven wall is sequentially photographed as oblique images so that the camera photographs at least the entire area in the direction of the oven length, and the condition of the oven wall inside the coke oven chamber is observed based on information on the imaging position of the oblique image (position in the direction of the oven length) and image information obtained by converting the oblique image into a frontal image.

[0004] Patent Document 2 discloses a technique for capturing images of both wall surfaces of a coke oven chamber from directions inclined relative to the normal direction of the two walls using separate cameras, and recognizing the type of damage occurring on both wall surfaces by referring to image data given as perspective images of the wall surfaces. Patent Document 3 discloses a technique for correlating the image of the inner wall surface with the oven width at the capture position by frequency-modulating an output voltage corresponding to the distance to the inner wall surface of a coke oven chamber measured by a laser range finder into an audio band signal using a signal converter and recording the signal together with an image of the inner wall surface captured by a video camera.

[0005] In this way, by inserting a heat-resistant camera into the furnace and using the image captured of the furnace wall to observe the furnace wall of the carbonization chamber, it is possible to observe the furnace wall even in the center of the furnace, allowing workers to perform their inspection work safely. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 3-105195 [Patent Document 2] JP 2001-3058 A [Patent Document 3] JP 2001-11465 A Summary of the Invention [Problem to be solved by the invention]

[0007] Coke is pushed out at a high frequency, once every 10 minutes, and the camera is inserted into the high-temperature coke chamber. When observing the coke chamber every time coke is pushed out, smoke may be generated in the chamber and the camera may be exposed to the smoke. For this reason, the camera gradually becomes dirty every time the pusher is inserted into the coke chamber, and the clarity of the captured image is lost due to the dirt on the camera's observation window. Since the camera's observation window is usually cleaned only on limited occasions, such as during furnace inspections, images are taken every time coke is pushed out with the dirt still on the camera's observation window, which may result in unclear captured images.

[0008] Therefore, the present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to provide an image processing device, an image processing method, and a program that can reduce the blur caused by dirt on the observation window of the imaging device being reflected in the captured image, and obtain clearer images of the inside of the furnace. [Means for solving the problem]

[0009] In order to solve the above problems, according to one aspect of the present invention, there is provided an image processing device that processes images captured inside a furnace, the image processing device having: a time representative image generating unit that generates a time representative image consisting of representative values ​​of pixel values ​​at the same pixel position from a plurality of images captured sequentially by an imaging device moving inside the furnace of the furnace wall; a filter generating unit that generates a filter array for correcting changes in pixel values ​​due to disturbances when the image is captured by the imaging device, based on each pixel value of the time representative image and the pixel value of the pixel position in the time representative image that is most likely to have no disturbances when the image is captured by the imaging device; and a correction processing unit that corrects the image captured by the imaging device using the filter array.

[0010] The filter generation unit may generate a filter array in which, for each pixel value of the time representative image, a pixel value at a pixel position that is most likely to be free of disturbance when the image is captured by the imaging device is divided by the pixel value.

[0011] Alternatively, the filter generation unit may generate a filter array in which, for each pixel value of the time representative image, a value is arranged by subtracting the pixel value from the pixel value of the pixel position that is most likely to be free of disturbance when the image is captured by the imaging device.

[0012] The image processing device may further include a filter correction coefficient setting unit that pre-sets a filter correction coefficient for correcting the filter array for each of a plurality of classes into which images are classified, and the correction processing unit may identify a class to which an image captured by the imaging device belongs, and correct the image using a correction filter array obtained by correcting the filter array based on the filter correction coefficient corresponding to the identified class.

[0013] In addition, in order to solve the above problem, according to another aspect of the present invention, there is provided an image processing device for generating a furnace wall observation image for observing the inside of a furnace, the image processing device having: a time representative image generating unit for generating a furnace wall front-view time representative image consisting of representative values ​​of pixel values ​​at the same pixel position from a plurality of furnace wall front-view images obtained by converting at least a portion of each of a plurality of oblique views of the furnace wall sequentially captured by an imaging device moving inside the furnace into a front-view image; a filter generating unit for generating a filter array for correcting changes in pixel values ​​due to disturbances when the image is captured by the imaging device based on each pixel value of the furnace wall front-view time representative image and the pixel value of the pixel position in the furnace wall front-view time representative image that is most likely to have no disturbances when the image is captured by the imaging device; and a correction processing unit for correcting the furnace wall front-view image using the filter array to generate a corrected furnace wall front-view image.

[0014] The image processing device may have an inter-frame movement amount calculation unit that calculates, by template matching processing, an inter-frame movement amount for linking a first corrected furnace wall front view image and a second corrected furnace wall front view image, which are two corrected furnace wall front view images including overlapping portions, for a plurality of corrected furnace wall front view images corrected by a correction processing unit, and an oven wall observation image generation unit that generates an oven wall observation image by linking the first corrected furnace wall front view image and the second corrected furnace wall front view image based on the inter-frame movement amount.

[0015] The image processing device may also have a filter correction coefficient setting unit that pre-sets a filter correction coefficient for correcting the filter array for each of a plurality of classes into which the front-view image of the oven wall is classified, and the correction processing unit may identify the class to which the front-view image of the oven wall belongs, and correct the front-view of the oven wall using a correction filter array in which the filter array is corrected based on the filter correction coefficient corresponding to the identified class, thereby generating a corrected front-view image of the oven wall.

[0016] Furthermore, in order to solve the above-mentioned problems, according to another aspect of the present invention, there is provided an image processing method for processing images captured inside a furnace, the image processing method including: generating a time representative image consisting of representative values ​​of pixel values ​​at the same pixel position from a plurality of images captured sequentially of the furnace wall by an imaging device moving inside the furnace; generating a filter array for correcting changes in pixel values ​​due to disturbances when the image is captured by the imaging device based on each pixel value of the time representative image and the pixel value of the pixel position in the time representative image which is most likely to have been free of disturbances when the image is captured by the imaging device; and correcting the image captured by the imaging device using the filter array.

[0017] In addition, in order to solve the above-mentioned problems, according to another aspect of the present invention, there is provided a program for image processing of images captured inside a furnace, which causes a computer to function as: a time representative image generating unit that generates a time representative image consisting of representative values ​​of pixel values ​​at the same pixel position from a plurality of images captured sequentially of the furnace wall by an imaging device moving inside the furnace; a filter generating unit that generates a filter array for correcting changes in pixel values ​​due to disturbances when the image is captured by the imaging device, based on each pixel value of the time representative image and the pixel value of the pixel position in the time representative image that is most likely to have not experienced disturbances when the image is captured by the imaging device; and a correction processing unit that corrects the image captured by the imaging device using the filter array. Effect of the Invention

[0018] As described above, according to the present invention, the captured image is corrected using a filter array for correcting changes in pixel values ​​due to external disturbances when the image is captured by the imaging device, thereby reducing blurring caused by dirt on the observation window of the imaging device that is reflected in the captured image, and a clearer image of the inside of the furnace can be obtained. [Brief description of the drawings]

[0019] [Figure 1] 1 is a schematic diagram showing a configuration example of a furnace wall observation device according to a first embodiment of the present invention. FIG. [Diagram 2] 2 is a functional block diagram showing a configuration example of an image processing device according to the embodiment. FIG. [Diagram 3] 4 is a flowchart showing the overall flow of an image processing method according to the embodiment. [Figure 4] 13 is a flowchart illustrating an example of a time averaged image generating process. [Diagram 5] 11 is a flowchart showing a process of generating a multiplication filter, which is an example of a filter generation process. [Figure 6] 11 is a flowchart showing an addition filter generation process, which is an example of a filter generation process. [Figure 7] FIG. 11 is a functional block diagram showing an example of the configuration of an image processing device according to a second embodiment of the present invention. [Figure 8] 4 is a flowchart showing the overall flow of an image processing method according to the embodiment. [Figure 9] 13 is a flowchart illustrating an example of a left and right region calculation process. [Figure 10] FIG. 13 is a schematic diagram illustrating an example of a time-averaged image. [Figure 11] FIG. 11 is an explanatory diagram illustrating a template matching process in a time averaged image. [Figure 12] 13 is a flowchart illustrating an example of a filter generation process. [Figure 13] 13 is a flowchart illustrating an example of an inter-frame movement amount calculation process. [Figure 14]11 is an explanatory diagram for explaining a process of performing perspective conversion on an oven wall oblique image to obtain an oven wall normal image. FIG. [Figure 15] FIG. 13 is an explanatory diagram showing the relationship between a search area in a current frame and a template image A2. [Figure 16] 13 is a flowchart illustrating an example of an inter-frame movement amount estimation process. [Figure 17] 13 is a flowchart showing an example of an oven wall observation image generating process. [Figure 18] FIG. 11 is an explanatory diagram for explaining a furnace wall observation image generating process. [Figure 19] 11 is a flowchart showing an example of a filter correction coefficient determination process for determining filter correction coefficients for correcting a filter array. [Figure 20] 4 is a flowchart showing an image processing method according to the first embodiment shown in FIG. 3, in which a filter array is corrected. [Figure 21] 10 is a flowchart illustrating an example of a filter correction coefficient selection process. [Figure 22] 1 is an image showing an example of a time-averaged image. [Figure 23] 23 shows an image before and an image after the time-averaged image shown in FIG. 22 is corrected using a filter array generated from the time-averaged image. [Figure 24] 23 is an image showing an image before and after correction of a newly acquired image of the inside of a furnace using a filter array generated from the time-averaged image shown in FIG. 22. [Diagram 25] 1 is a block diagram showing an example of a hardware configuration of an information processing device that functions as an image processing device according to each embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and duplicated explanations will be omitted.

[0021] [1. First embodiment] [1-1. Observation of the carbonization chamber of a coke oven] First, a configuration example of an oven wall observation device that acquires an image of the oven wall of the coke oven chamber will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing a configuration example of an oven wall observation device according to this embodiment.

[0022] The coke oven 1 is a furnace for producing coke. Carbonization chambers and combustion chambers are arranged alternately in the upper part of the furnace body of the coke oven 1, and a heat regenerator is provided in the lower part. Coke is produced by burning combustion gas and combustion air, which have been preheated in the heat regenerators, in the combustion chamber, and using the generated heat to dry distill the coal charged in the carbonization chamber.

[0023] Fig. 1 shows the inside of the coke chamber 10 of the coke oven 1 as viewed from the width direction of the coke chamber (X direction). The coke chamber 10 is composed of two oven walls 11 facing each other in the width direction of the coke chamber, and Fig. 1 shows only one oven wall 11 at the back of the page. Coke produced by carbonizing coal charged in the coke chamber 10 is pushed out to the coke discharge side of the coke chamber 10 by a push-out ram 21 provided at one end of a push-out ram beam 20 moving in the pushing direction (Y direction), and is discharged from the coke chamber 10.

[0024] The extrusion Ram beam 20 in FIG. 1 is provided with two imaging devices 31 and 33 arranged in parallel in the height direction (Z direction) at the center of the width of the coke chamber 10. The imaging devices 31 and 33 function as oven wall observation devices that acquire oven wall images of the oven wall 11 of the coke chamber 10. The imaging devices 31 and 33 are equipped with a cooling mechanism or a heat-resistant mechanism so as to withstand the high-temperature environment in the coke chamber 10. The imaging devices 31 and 33 shown in FIG. 1 capture images of the oven wall 11 sequentially when the extrusion Ram beam 20 moves from the extruder side to the coke discharge side or when the extrusion Ram beam 20 moves from the coke discharge side to the extruder side, with the extruder side opposite to the coke discharge side as the imaging direction. This allows the inside of the coke chamber 10 to be observed every time the coke is extruded. The images acquired by the imaging devices 31 and 33 are output to an image processing device described later.

[0025] 1, two imaging devices 31 and 33 are installed, but the present invention is not limited to this example, and the number of imaging devices may be one or three or more. Also, the imaging devices 31 and 33 may be provided on a moving mechanism other than the extrusion Lamb beam 20 that can move along the extrusion direction inside the coking chamber 10.

[0026] [1-2. Image processing device] 1 are output to an image processing device 50 according to this embodiment. The image processing device 50 according to this embodiment is a device for processing an image of the oven wall 11 inside the coke chamber 10 of the coke oven 1, reducing dirt on the observation windows of the imaging devices 31 and 33 that is reflected in the image, and providing a clearer image of the inside of the oven.

[0027] 2 shows an example of the configuration of an image processing device 50 according to this embodiment. The image processing device 50 according to this embodiment includes a time representative value image generating unit 51, a filter generating unit 53, and a correction processing unit 55, as shown in FIG.

[0028] The time representative image generating unit 51 generates a time representative image consisting of a representative value of pixel values ​​at the same pixel position from a plurality of images of the oven wall 11 sequentially captured by the imaging devices 31, 33 moving inside the oven. The images of the oven wall captured by the imaging devices 31, 33 are recorded, for example, in a storage unit (not shown). The time representative image generating unit 51 generates a time representative image using a plurality of images acquired in one coke push. The time representative image is an image that represents the luminance distribution of the images captured by the imaging devices 31, 33 by the representative value of pixel values ​​at the same pixel position in the plurality of images. The representative value of pixel values ​​is a value that indicates the central tendency of pixel values, and is, for example, the average value, the median value, the mode value, etc. The time representative image generating unit 51 outputs the generated time representative image to the filter generating unit 53.

[0029] The filter generation unit 53 generates a filter array for correcting changes in pixel values ​​due to disturbances when the image is captured by the imaging devices 31, 33, based on each pixel value of the time representative value image and pixel values ​​of healthy pixel positions in the time representative value image that are most likely to be free of disturbances when the image is captured by the imaging devices 31, 33. The filter array is used to restore images captured by the imaging devices 31, 33 that have become blurred due to disturbances such as dirt on the observation windows of the imaging devices 31, 33.

[0030] Since the inside of the furnace is generally hot, if the observation windows of the imaging devices 31 and 33 are not dirty, the pixel values ​​of all pixel positions in the time representative image will be almost the same. However, if the observation windows of the imaging devices 31 and 33 are partially dirty, the pixel values ​​of the dirty positions in the time representative image will be smaller than the pixel values ​​of healthy positions (i.e., positions where the observation windows of the imaging devices 31 and 33 are not dirty). Therefore, the maximum value of the pixel values ​​in the time representative image (hereinafter also referred to as the "maximum brightness value") is almost equal to the pixel value when the dirty positions in the time representative image are not dirty. Even if there is no position in the time representative image that is completely free of dirt, the maximum brightness value in the time representative image is considered to be closest to the brightness value when the image is healthy. In other words, the maximum brightness value in the time representative image can be said to be the pixel value of a healthy pixel position that is most likely to be free of disturbances when the imaging devices 31 and 33 are capturing images. Therefore, the filter generation unit 53 generates a filter array based on each pixel value of the time representative image and the maximum luminance value to correct the change in pixel value due to the change in the amount of light caused by disturbances such as dirt on the observation windows of the imaging devices 31 and 33. The details of the filter generation process will be described later. The filter generation unit 53 outputs the generated filter array to the correction processing unit 55.

[0031] The correction processing unit 55 uses the filter array to correct the images of the inside of the furnace captured by the imaging devices 31, 33. By correcting the images of the inside of the furnace captured by the imaging devices 31, 33 using the filter array, pixel values ​​that have decreased due to disturbances such as dirt on the observation windows of the imaging devices 31, 33 are restored. This makes the images of the inside of the furnace captured by the imaging devices 31, 33 clearer.

[0032] [1-3. Image processing method] Next, the image processing method by the image processing device 50 according to this embodiment will be described in detail with reference to Figs. 3 to 6. Fig. 3 is a flowchart showing the overall flow of the image processing method according to this embodiment. Details of each process in the flowchart will be described with reference to Figs. 4 to 6. Below, a case will be described in which the maximum luminance value of the time representative value image is used as the pixel value of the healthy pixel position in which the probability that no disturbance occurs during image capture by the imaging devices 31 and 33 in the time representative value image is the highest. Also, below, a case will be described in which a filter array for correcting an image is generated from multiple images acquired by one imaging device (for example, the imaging device 31), and an image captured by the imaging device is corrected using the filter array. When images inside the coking chamber can be acquired by multiple imaging devices, the image processing method described below may be performed for each imaging device.

[0033] (S1: Time representative value image generation process) First, the time representative value image generating unit 51 calculates a representative value of pixel values ​​for each pixel at the same pixel position from a plurality of images of the oven wall 11 captured sequentially by an imaging device (e.g., the imaging device 31) moving inside the oven, and generates a time representative value image composed of pixels having the calculated representative value as a pixel value (S1). The images of the oven wall captured by the imaging device are recorded, for example, in a storage unit (not shown). The time representative value image generating unit 51 uses a plurality of images acquired in one coke push to generate a time representative value image represented by a value indicating a central tendency of pixel values, such as an average value, a median value, or a mode value.

[0034] FIG. 4 is a flowchart showing an example of a time-averaged image generation process, which is an example of a time-representative image generation process. The time-averaged image generation process is a process for generating a time-averaged image, which is a time-representative image using an average value as a representative value. As shown in FIG. 4, the time-representative image generation unit 51 first sets each element of the time-averaged image array, which is a two-dimensional array that stores pixel values ​​of the time-averaged image, and a counter that counts the number of frames to an initial value (S11). Note that a frame means each image that is sequentially acquired in one coke extrusion. Each element of the time-averaged image array corresponds one-to-one to each pixel that constitutes a frame, and a value of, for example, "0" is stored as an initial value. After step S11, the time-representative image generation unit 51 starts reading (acquiring) an image (frame) of the oven wall captured by an imaging device and recorded in a storage unit or the like (S12). In step S12, the frames are read in chronological order (for example, from the extruder side to the coke discharge side).

[0035] The time representative image generating unit 51 executes frame reading in step S12 and determines whether or not the frame reading has been successful (S13). If the frame reading has been successful (S13: YES), each pixel value of the frame is added to the element of the time average image array corresponding to each pixel (S14). That is, every time a new frame is read, the pixel value of each pixel constituting the read frame is added to each corresponding element of the time average image array. Then, the time representative image generating unit 51 adds 1 to the counter value (S15), returns to the process of step S12, and reads the next frame. The processes of steps S12 to S15 are repeatedly executed until reading of all frames is completed.

[0036] On the other hand, if no frame can be read in step S13 (S13: NO), since reading up to the final frame has been completed, the pixel values ​​stored in each element of the time-averaged image array at that point (i.e., the sum of pixel values ​​at the same pixel position in each frame) are divided by the counter value to generate a time-averaged image (S16). The counter value corresponds to the number of frames read. In this way, a time-averaged image is generated, which is an average image of images acquired during one movement from the pusher side to the coke discharge side. The time-representative image generating unit 51 outputs the generated time-representative image to the filter generating unit 53.

[0037] (S3: Filter generation process) Next, the filter generating unit 53 generates a filter array based on each pixel value and the maximum luminance value of the time representative image (S3). The filter array is used to restore an image captured by the imaging device that has become blurred due to disturbances such as dirt on the observation window of the imaging device. The filter array corrects changes in pixel values ​​with respect to changes in the amount of light caused by disturbances such as dirt on the observation window of the imaging device. Specific examples of the filter array include a multiplication filter formed by dividing the maximum luminance value of the time representative image by the pixel value of the time representative image, and an addition filter formed by subtracting the pixel value of the time representative image from the maximum luminance value of the time representative image. A specific example of the filter array generating process will be described below with reference to Figs. 5 and 6.

[0038] (a) Multiplication filter generation process First, a multiplication filter generation process, which is an example of a filter generation process according to the present embodiment, will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the multiplication filter generation process. In the process shown in Fig. 5, a time average image is used as a time representative value image.

[0039] First, the filter generating unit 53 generates a mask array that defines a pixel value conversion region in which pixel values ​​are converted (S31a). The pixel value conversion region is a region in an image in which correction is performed using a filter array. An image of the inside of a furnace captured by an imaging device may include regions other than the furnace wall portion, and regions other than the furnace wall portion may be unnecessary. Therefore, a necessary region in an image may be set as a pixel value conversion region. The pixel value conversion region may be, for example, a region specified by a user, or may be a region in an image that is set in advance. A mask array is generated based on the set pixel value conversion region.

[0040] Next, the filter generating unit 53 obtains the maximum luminance value in the pixel value conversion region of the time average image (S32a). The pixel having the maximum luminance value is considered to be a pixel where the observation window is definitely free of dirt (healthy) at that position, or a pixel closest to the pixel value in the case of a healthy state. Therefore, the filter generating unit 53 identifies a pixel having the maximum luminance value in the pixel value conversion region of the time representative value image generated in step S1 of FIG. 3, and obtains the maximum luminance value. Then, the filter generating unit 53 divides the pixel value of each pixel in the entire time average image by the maximum luminance value, and stores the result in a temporary array (S33a). Here, the temporary array has elements that correspond one-to-one to each pixel that constitutes the time average image.

[0041] Thereafter, the filter generating unit 53 replaces the values ​​of elements of the temporary array corresponding to pixels constituting the time-averaged image in an area other than the pixel value conversion area determined by the mask array with "1" (S34a), and then calculates the reciprocal of the value of each element of the primary array to generate a multiplication filter (S35a). The process of step S34a is performed to prevent the calculation from becoming impossible when calculating the reciprocal in step S35a. The multiplication filter generated in step S35a is expressed by the following formula (1).

[0042]

number

[0043] It should be noted that when the captured image is in color (for example, an RGB image), calculations such as addition, subtraction, multiplication, and division of pixel values ​​are performed separately for each color (for example, "R", "G", and "B").

[0044] If there is no disturbance such as dirt on the observation window of the imaging device, the inside of the furnace is generally at a high temperature, so the pixel values ​​of all pixel positions in the time representative image should be approximately equal to the maximum brightness value. If we assume that there is a proportional relationship between the change in pixel value and the change in light amount due to the adhesion of dirt, the pixel value of the dirty position will be reduced by the following formula (2) compared to the pixel value of the healthy position (assumed to be the maximum brightness value here).

[0045]

number

[0046] The multiplication filter is expressed by the reciprocal of the pixel value decrement rate in the above formula (2). Therefore, if there is a disturbance such as dirt on the observation window of the imaging device, the pixel value at that position can be corrected (multiplied) by the multiplication filter expressed by the above formula (1) to restore the pixel value that has decreased due to the disturbance. On the other hand, if there is no disturbance such as dirt on the observation window of the imaging device, the pixel value at that position does not need to be corrected, but the pixel value at that position is approximately equal to the maximum luminance value. Therefore, the value of the multiplication filter expressed by the above formula (1) is approximately 1, and correction (multiplication) by this multiplication filter does not result in correction.

[0047] (b) Addition filter generation process Next, an addition filter generation process, which is an example of the filter generation process according to the present embodiment, will be described with reference to Fig. 6. Note that the addition filter generation process is executed when an addition filter is used instead of the above multiplication filter. Fig. 6 is a flowchart showing the addition filter generation process. In the process shown in Fig. 6, a time average image is used as a time representative value image.

[0048] First, the filter generating unit 53 generates a mask array that defines a pixel value conversion region in which pixel values ​​are converted (S31b). The process of step S31b may be performed in the same manner as step S31a in FIG.

[0049] Next, the filter generating unit 53 obtains the maximum luminance value in the pixel value conversion domain of the time average image (S32b). The process of step S32b may be performed in the same manner as step S32a in FIG.

[0050] Then, filter generation unit 53 stores the value obtained by subtracting each pixel value of the time-averaged image from the maximum brightness value in the filter array (S33b), and then replaces the values ​​of elements of the filter array that correspond to pixels that constitute the time-averaged image and are in an area other than the pixel value conversion area determined by the mask array with "0" (S34b). In this way, an addition filter is generated. The addition filter generated in step S34b is expressed by the following equation (3).

[0051]

number

[0052] If there is no disturbance such as dirt on the observation window of the imaging device, the inside of the furnace is generally hot, so the pixel values ​​of all pixel positions in the time representative image should be approximately equal to the maximum brightness value. If we assume that there is additivity in the change in pixel value relative to the change in light amount due to the adhesion of dirt, the pixel value of the dirty position will be reduced by the following formula (4) compared to the pixel value of the healthy position (assumed to be the maximum brightness value here).

[0053]

number

[0054] The additive filter is used to restore the pixel value decrement in the above formula (4). If there is a disturbance such as dirt on the observation window of the imaging device, the pixel value at that position can be corrected (added) using the additive filter expressed by the above formula (3) to restore the pixel value that was reduced by the disturbance. On the other hand, if there is no disturbance such as dirt on the observation window of the imaging device, the pixel value at that position does not need to be corrected, but the pixel value at that position is approximately equal to the maximum luminance value. For this reason, the value of the additive filter expressed by the above formula (3) is approximately 0, and correction (addition) using this additive filter does not result in correction.

[0055] The filter generation unit 53 generates a filter array by the process shown in FIG. 5 or FIG.

[0056] (S5: Correction process) Returning to the explanation of FIG. 3, when the filter array is generated in step S3, the correction processing unit 55 corrects the image of the inside of the furnace using the filter array.

[0057] For example, when a multiplication filter represented by the above formula (1) is used as the filter array, the value of each pixel of the corrected image is expressed by the following formula (5-1): When an addition filter represented by the above formula (3) is used as the filter array, the value of each pixel of the corrected image is expressed by the following formula (5-2):

[0058]

number

[0059] In this way, by correcting the image of the inside of the furnace captured by the imaging device using the filter array, pixel values ​​that have decreased due to disturbances such as dirt on the observation window of the imaging device are restored, making it possible to make the image of the inside of the furnace captured by the imaging device clearer.

[0060] The image processing device 50 according to the first embodiment and the image processing method using the image processing device have been described above. According to this embodiment, a time representative image consisting of representative values ​​of pixel values ​​at the same pixel position is generated from a plurality of images of the furnace wall captured sequentially by an imaging device moving inside the furnace, and a filter array for correcting changes in pixel values ​​due to disturbances when the imaging device captures an image is generated based on each pixel value of the time representative image and the pixel value (e.g., maximum brightness value) of a healthy pixel position in the time representative image that is most likely to be free of disturbances when the imaging devices 31 and 33 capture the image, and the image captured by the imaging device is corrected using the filter array. This restores pixel values ​​that have decreased due to disturbances such as dirt on the observation window of the imaging device, making it possible to make the image of the furnace interior obtained by the captured image clearer.

[0061] [2. Second embodiment] Next, an image processing device 100 for processing an image captured inside a coke oven according to a second embodiment of the present invention and an image processing method using the same will be described.

[0062] [2-1. Image processing device] The image processing device 100 of this embodiment is a device that generates an oven wall observation image by correcting multiple oven wall normal images obtained by perspective conversion of an oblique image of the oven wall 11 inside the carbonization chamber 10 of a coke oven 1 using the filter array described in the first embodiment above, and then concatenating them.

[0063] The images acquired by the imaging devices 31 and 33 are not normal images obtained by setting the imaging devices 31 and 33 directly opposite the oven wall 11 and orienting the imaging direction to the front of the oven wall 11, but are oblique images obtained by imaging the oven wall 11 in an oblique state by setting the imaging devices 31 and 33 with the imaging direction as the extrusion direction. In addition, in each oblique image acquired while moving the imaging devices 31 and 33 in the extrusion direction of the coking chamber 10, the parts farther away from the positions of the imaging devices 31 and 33 at the time of capturing each oblique image become unclear, so it is difficult to clearly confirm the entire oven wall 11 shown in each oblique image. However, the oven wall 11 near the position in the extrusion direction when each oblique image is captured can be clearly confirmed from each oblique image.

[0064] Therefore, the image processing device 100 according to this embodiment extracts a portion where the oven wall can be clearly confirmed from the oblique image captured by the imaging devices 31 and 33, and generates an oven wall front view image by performing perspective conversion on the extracted partial image (oven wall oblique image). Then, the image processing device 100 calculates an inter-frame shift amount for connecting multiple oven wall front view images based on the generated oven wall front view image, and connects the multiple oven wall front view images using the calculated inter-frame shift amount to generate an oven wall observation image showing the entire oven wall 11.

[0065] In this way, in the image processing device 100 according to this embodiment, the amount of movement between frames is calculated based on the acquired oven wall oblique image without using the position information of the imaging devices 31, 33 in the extrusion direction, and the oven wall front view images are connected using the calculated amount of movement between frames to generate an oven wall observation image. This makes it possible to obtain an image that more accurately and smoothly represents the entire oven wall 11 without being affected by subtle deviations in the field of view of the imaging devices 31, 33 due to vibrations or the like caused when the imaging devices 31, 33 move. In addition, since the oven wall front view images that constitute the oven wall observation image are corrected by the filter array, a clearer oven wall observation image can be generated.

[0066] 7 shows an example of the configuration of the image processing device 100 according to this embodiment. As shown in FIG. 7, the image processing device 100 according to this embodiment includes an image acquisition unit 110, a preprocessing unit 120, an inter-frame movement amount calculation unit 130, an inter-frame movement amount estimation unit 140, a furnace wall observation image generation unit 150, and a filter generation unit 160.

[0067] The image acquisition unit 110 acquires the oven wall perspective image Ip captured by the imaging devices 31 and 33 from the captured image storage unit 310 of the storage device 300. The captured image storage unit 310 is a storage unit that stores the oven wall perspective image Ip captured by the imaging devices 31 and 33, and is provided in the storage device 300 constituted by, for example, a server. The storage device 300 may be provided in the image processing device 100.

[0068] The imaging devices 31 and 33 continuously capture images of the oven wall 11 during one movement from the pusher side to the coke discharge side. As a result, the imaging devices 31 and 33 obtain a video of an imaging target portion of the oven wall 11, such as the entire oven wall 11, which is composed of a plurality of oven wall oblique images Ip (frames) captured in an oblique state of the oven wall 11. A part of the oven wall 11 is captured in each oven wall oblique image Ip, and a part of the oven wall 11 captured in the oven wall oblique image Ip includes a part overlapping with a part of the oven wall 11 captured in another oven wall oblique image Ip. Note that it is sufficient that at least two oven wall oblique images Ip that are consecutive in the imaging order have an overlapping part, and two discontinuous oven wall oblique images Ip separated by one or more frames may have an overlapping part.

[0069] A plurality of oven wall perspective images Ip captured by the imaging devices 31 and 33 are sequentially recorded in the captured image storage unit 310. Specifically, the imaging devices 31 and 33 are connected to the storage device 300 wirelessly or wired so as to be able to communicate with each other, and the images are transmitted from the imaging devices 31 and 33 to the storage device 300. The image acquisition unit 110 outputs the oven wall perspective images Ip acquired from the captured image storage unit 310 to the preprocessing unit 120, the inter-frame movement amount calculation unit 130, and the oven wall observation image generation unit 150.

[0070] The pre-processing unit 120 performs pre-processing necessary for generating an oven wall normal image Is from a plurality of oven wall perspective images Ip captured by the imaging devices 31, 33 while the imaging devices 31, 33 are moved from the pusher side to the coke discharge side. When the pre-processing unit 120 receives the oven wall perspective images Ip from the image acquisition unit 110, the pre-processing unit 120 performs processing for appropriately cutting out a portion representing the two oven walls 11 of the coking chamber 10 captured in each of the oven wall perspective images Ip. Specifically, the pre-processing unit 120 obtains a symmetry axis C (symmetry axis C in FIG. 7) for specifying an area representing the two oven walls 11 on the left and right of the coking chamber 10 (hereinafter, also referred to as the "right area" and the "left area" of the oven wall, or collectively as the "left and right areas") from the oven wall perspective images Ip. The process of obtaining the symmetry axis C by the pre-processing unit 120 will be described in detail later. The pre-processing unit 120 outputs information for setting the obtained symmetry axis C to the inter-frame movement amount calculation unit 130 and the oven wall observation image generation unit 150.

[0071] The frame-to-frame movement amount calculation unit 130 acquires a plurality of oven wall front-view images Is using the preprocessing results by the preprocessing unit 120. Then, for the obtained plurality of oven wall front-view images Is, the first oven wall front-view image Is1 and the second oven wall front-view image Is2, which are two oven wall front-view images Is including overlapping parts, calculates the frame-to-frame movement amount L between the first oven wall front-view image Is1 and the second oven wall front-view image Is2 by template matching processing. The frame-to-frame movement amount L represents the amount of deviation between the first oven wall front-view image Is1 and the second oven wall front-view image Is2. The frame-to-frame movement amount L is used to determine the position at which the parts cut out from the first oven wall front-view image Is1 and the second oven wall front-view image Is2 are connected by the oven wall observation image generation processing described later. The inter-frame movement amount calculation unit 130 outputs the calculated inter-frame movement amount L to the inter-frame movement amount estimation unit 140 and the oven wall observation image generation unit 150.

[0072] The inter-frame movement amount estimating unit 140 calculates an inter-frame movement amount estimated value E for correcting the inter-frame movement amount L calculated by the inter-frame movement amount calculating unit 130. The inter-frame movement amount L may not be calculated correctly due to the influence of noise such as smoke captured in the captured image. The inter-frame movement amount estimating unit 140 calculates the inter-frame movement amount estimated value E as a correction value for correcting the inter-frame movement amount L calculated by the inter-frame movement amount calculating unit 130 to a more likely value when the inter-frame movement amount L is outside the range of a threshold. The inter-frame movement amount estimating unit 140 outputs the calculated inter-frame movement amount estimated value E to the oven wall observation image generating unit 150.

[0073] The oven wall observation image generating unit 150 connects the oven wall front-view images Is to generate an oven wall observation image I. When the inter-frame movement amount L calculated by the inter-frame movement amount calculating unit 130 is within a preset threshold range, the oven wall observation image generating unit 150 connects the first oven wall front-view image Is1 and the second oven wall front-view image Is2 based on the inter-frame movement amount. On the other hand, when the inter-frame movement amount L calculated by the inter-frame movement amount calculating unit 130 is outside the threshold range, the oven wall observation image generating unit 150 connects the first oven wall front-view image Is1 and the second oven wall front-view image Is2 based on the inter-frame movement amount estimated value E calculated by the inter-frame movement amount estimating unit 140.

[0074] The oven wall observation image generating unit 150 appropriately sets the inter-frame movement amount L between the first oven wall front image Is1 and the second oven wall front image Is2 for all oven wall front images Is, and connects them to generate an oven wall observation image I. The oven wall observation image generating unit 150 may output the generated oven wall observation image I to a display device 500 communicably connected to the image processing device 100. Alternatively, the oven wall observation image generating unit 150 may record the generated oven wall observation image I in the oven wall observation image storage unit 320 of the storage device 300.

[0075] The filter generating unit 160 generates a filter array used for correcting the oven wall normal view image in the inter-frame movement amount calculating unit 130 and the oven wall observation image generating unit 150. The filter array is for correcting the change in pixel value due to disturbance when the image is captured by the imaging devices 31 and 33, based on each pixel value of the time representative value image and the pixel value of a healthy pixel position in the time representative value image that is most likely to be free of disturbance when the image is captured by the imaging devices 31 and 33. The filter generation process by the filter generating unit 160 will be described in detail later.

[0076] The image processing device 100 according to the present embodiment has been described above. In the image processing device 100 according to the present embodiment, the conversion from the oven wall perspective image Ip to the oven wall normal image Is is performed by the frame-to-frame movement amount calculation unit 130 and the frame-to-frame movement amount estimation unit 140, respectively, but the present invention is not limited to this example. For example, the oven wall normal image Is generated by a specific one functional unit may be configured to be passed to another functional unit, such as passing the oven wall normal image Is generated by the frame-to-frame movement amount calculation unit 130 to the frame-to-frame movement amount estimation unit 140.

[0077] [2-2. Image processing method] The image processing method by the image processing device 100 according to this embodiment will be described in detail below with reference to Figs. 8 to 18. Fig. 8 is a flowchart showing the overall flow of the image processing method according to this embodiment. Details of each process in the flowchart will be described with reference to Figs. 9 to 18. In the following, a case will be described in which one oven wall observation image I is obtained from a plurality of oven wall perspective images Ip acquired by one imaging device (e.g., imaging device 31) during one movement from the pusher side to the coke discharge side. When images of the inside of the coke chamber can be acquired by a plurality of imaging devices, the image processing method described below may be performed for each imaging device.

[0078] (S10: Furnace wall oblique image acquisition process) In the image processing method according to the present embodiment, as shown in FIG. 8, first, the image acquisition unit 110 acquires a plurality of oven wall perspective images Ip recorded in the captured image storage unit 310 of the storage device 300 (S0). For example, in one movement from the pusher side to the coke discharge side, the imaging device (e.g., the imaging device 31) acquires a plurality of oven wall perspective images Ip. The oven wall perspective images Ip can also be considered as frames constituting a video. The image acquisition unit 110 acquires a plurality of oven wall perspective images Ip (frames) obtained by the imaging device (e.g., the imaging device 31) continuously capturing images of the oven wall 11 during one movement of the extrusion ram beam 20 from the captured image storage unit 310. The image acquisition unit 110 outputs the acquired plurality of oven wall perspective images Ip to the pre-processing unit 120.

[0079] (Pretreatment) Next, the preprocessing unit 120 performs a time average image generation process (S10) and a left and right region calculation process (S20) as preprocessing.

[0080] If the imaging devices 31 and 33 are exactly facing the extrusion direction, the oven wall 11 is photographed so that the center of the coking chamber kiln mouth is on the vertical central axis of the oven wall oblique image Ip, so the right and left regions of the oven wall 11 in the oven wall oblique image Ip can be set in advance. However, it is difficult in terms of hardware to make the imaging devices 31 and 33 exactly facing the extrusion direction. Therefore, in this embodiment, by performing a time-averaged image generation process and a left and right region calculation process by the pre-processing unit 120, even if there is a slight deviation between the vertical central axis of the oven wall oblique image Ip and the center of the coking chamber kiln mouth, the deviation is absorbed in software.

[0081] (S10: Time-averaged image generation process) In the time-averaged image generation process, a time-averaged image used in the left-right region calculation process is generated. In order to identify the left-right region in the left-right region calculation process, the time-averaged image is used without using a specific oven wall perspective image Ip because the left-right symmetry of the partial image near the coking chamber mouth of the time-averaged image is higher than that of the specific oven wall perspective image Ip. The partial image near the coking chamber mouth of the specific oven wall perspective image Ip includes reflections of things other than the oven wall, such as sparks, and reflections of joints that are not left-right symmetrical. For this reason, the partial image near the coking chamber mouth of the specific oven wall perspective image Ip may not be suitable for use in estimating the axis of symmetry by matching using a left-right inverted image (template image A1) in the left-right region calculation process described later. Therefore, in this embodiment, a time-averaged image is generated and the left-right region calculation process described later is performed. The time-averaged image generation process in step S10 may be performed by the pre-processing unit 120, for example, by the process shown in FIG. 4.

[0082] (S20: Left and right area calculation process) Next, the pre-processing unit 120 performs left and right region calculation processing. In the left and right region calculation processing, regions representing the two oven walls 11 on the left and right sides of the coking chamber 10 (hereinafter, also referred to as the "right region" and the "left region" of the oven walls, or collectively as the "left and right regions") are identified from each oven wall perspective image Ip (frame). An example of the left and right region calculation processing is shown in FIG. 9.

[0083] The pre-processing unit 120 first sets and cuts out a partial area A0 to be a template image A1 from the time-averaged image obtained in the time-averaged image generation process of step S10 (S201). The time-averaged image Ia shows two oven walls 11R and 11L on the left and right sides of the captured coking chamber 10, as shown in FIG. 10. The oven bottom 13 of the coking chamber and the coking chamber kiln port 15 on the extruder side of the coking chamber 10 may also appear. For such a time-averaged image Ia, the partial area A0 of the time-averaged image Ia is flipped left and right as shown in the upper part of FIG. 11 to set the template image A1. The partial area A0 of the time-averaged image Ia to be the template image A1 is an area including a part of the oven wall that is smaller in size than the time-averaged image Ia and can be arbitrarily set by the user.

[0084] The template image A1 is set to include an image of the coke oven chamber mouth 15. For example, as shown in the upper part of FIG. 11, the width of the template image A1 is set to a width obtained by reducing the width W of the time-averaged image Ia by a length x1 from each of the left end and the right end of the time-averaged image Ia. The length x1 can be obtained in advance from the correspondence between the width W of the time-averaged image Ia and the position near the center of the coke oven chamber mouth 15. Note that the height of the template image A1 may be set as appropriate since it has little influence on the left and right region calculation process.

[0085] Next, the preprocessing unit 120 horizontally flips the partial region A0 cut out from the time-averaged image Ia in step S201 to obtain a template image A1 as shown on the right side of FIG. 11 (S203).

[0086] Then, the preprocessing unit 120 performs a template matching process to identify an image region that matches the template image A1 (S205).

[0087] The template matching process is an image processing method for obtaining an image region with the highest similarity to the template image in the search target image. Specifically, when the size of the search target image is N [pixels] × M [pixels] and the size of the template image is n [pixels] × m [pixels] (0 < n ≤ N, 0 < m ≤ M), a two-dimensional array of (N - n + 1) × (M - m + 1) is obtained as the template matching result R by the template matching process. In the (i, j) element (0 ≤ i < N - n + 1, 0 ≤ j < M - m + 1) of the template matching result R, a numerical value representing the similarity between the image region (image size: n × m, pixel position (i, j) of the upper left vertex) of the search target image and the template image is stored. If the element in which the highest similarity numerical value is stored in each element of the template matching result R is the (ig, jh) element (0 ≤ ig < N - n + 1, 0 ≤ jh < M - m + 1), the image region in the search target image with the pixel position of the upper left vertex being (ig, jh) and the image size being n × m is obtained as the image region with the highest similarity to the template image.

[0088] The pre-processing unit 120 determines an axis of symmetry C along which the left and right regions are symmetrical when the time-averaged image Ia is divided in the width direction from the position of the image region in the time-averaged image Ia that has the highest similarity to the template image A1, obtained by the template matching process (S207). For example, as shown in the lower part of FIG. 11, the position x of the axis of symmetry C is expressed as x=(s+W-x1) / 2.

[0089] Note that the length s is the length from one end (e.g., the left end) of the time averaged image Ia in the width direction to one end of the image area where the template image A1 matches when the position of the image area where the template image A1 matches in the time averaged image Ia is identified.

[0090] The symmetry axis C determined in step S207 is used to divide the oven wall perspective image Ip (frame) into left and right regions. By determining the symmetry axis C in this manner, the oven wall perspective image Ip (frame) can be divided more accurately into left and right regions. In addition, the partial region A0 cut out from the time-averaged image Ia is flipped left and right to create a template image A1, and the template matching process is performed, so that the symmetry axis C can be estimated by a simple algorithm.

[0091] (S25: Filter generation process) Returning to the explanation of Fig. 8, upon completion of the time average image generation process (S10) and the left and right region calculation process (S20), the filter generation unit 160 performs a filter generation process (S25). Fig. 12 is a flowchart showing a multiplication filter generation process, which is an example of the filter generation process.

[0092] 12, first, the filter generating unit 160 divides the time-averaged image generated in step S10 into a left region and a right region (S251). Then, the filter generating unit 160 performs perspective transformation on each of the left region and the right region to obtain a normal image of each region (S252).

[0093] Next, the filter generating unit 160 generates a mask array that defines a pixel value conversion region in each of the left and right normal images (S253). The pixel value conversion region is a region in each of the right and left normal images where correction is performed using the filter array. The pixel value conversion region may be, for example, a region designated by the user, or may be a region in the normal image that has been set in advance. A mask array is generated based on the set pixel value conversion region.

[0094] Then, the filter generating unit 160 obtains the maximum luminance value as the pixel value of the healthy pixel position that is most likely to have no disturbance during image capture by the imaging devices 31 and 33 in the pixel value conversion region of each of the left and right normal images (S254). The pixel having the maximum luminance value is considered to be a pixel where the observation window is certainly free of dirt (healthy) at that position, or a pixel closest to the pixel value when healthy. Therefore, the filter generating unit 160 identifies the pixel having the maximum luminance value in the pixel value conversion region of the time-averaged image generated in step S253, and obtains the maximum luminance value. Then, the filter generating unit 160 divides the pixel value of each pixel of the normal images of the left and right regions by the maximum luminance value, and stores the result in a temporary array (S255). Here, the temporary array has elements that correspond one-to-one to each pixel that constitutes the time-averaged image.

[0095] Thereafter, the filter generating unit 160 replaces the values ​​of elements of the temporary array corresponding to pixels constituting the time-averaged image in an area other than the pixel value conversion area determined by the mask array with "1" (S256), and then calculates the reciprocal of the value of each element of the primary array to generate a multiplication filter (S257). The process of step S256 is performed to prevent the calculation from becoming impossible when calculating the reciprocal in step S257. The multiplication filter generated in step S257 is expressed as in the above formula (1).

[0096] 12, a process of generating a multiplication filter having a value obtained by dividing the maximum luminance value of the time representative image by the pixel value of the time representative image has been described, but the present invention is not limited to this example. For example, an addition filter having a value obtained by subtracting the pixel value of the time representative image from the maximum luminance value of the time representative image, as described in the first embodiment, may be generated as a filter array.

[0097] 12, the filter array is generated from a normal image obtained by performing perspective transformation on the left and right regions of the time-averaged image, but the present invention is not limited to this example. For example, the filter array may be generated based on the left and right regions (i.e., oblique image) of the time-averaged image before perspective transformation generated in step S251 of FIG.

[0098] (S30: Inter-frame movement amount calculation process) Returning to the explanation of Fig. 8, upon completion of the filter generation process (S25), the inter-frame movement amount calculation section 130 performs an inter-frame movement amount calculation process (S30). Fig. 13 is a flowchart showing an example of the inter-frame movement amount calculation process.

[0099] 13, the inter-frame movement amount calculation unit 130 first sets a movement amount storage array, which is a two-dimensional array for storing the calculated inter-frame movement amount L, to an initial value (S301). For example, a value of "0" is stored as the initial value. The movement amount storage array is prepared for a left region and a right region obtained by dividing the oven wall perspective image Ip.

[0100] As described above, the inter-frame movement amount L is the amount of shift between the first oven wall front image Is1 and the second oven wall front image Is2. That is, the inter-frame movement amount L is the amount by which the second oven wall front image Is2 is moved so that the parts of the oven wall overlapping in the oven wall front images Is are perfectly overlapped from the state in which the first oven wall front image Is1 and the second oven wall front image Is2 having the same image size are superimposed. Since the inter-frame movement amount L is an amount representing the shift between the two oven wall front images Is, it can be expressed by the number of pixels. Since the left-right direction of the oven wall front image Is coincides with the extrusion direction of the coking chamber, the inter-frame movement amount L is represented by the number of pixels by which the second oven wall front image Is2 is shifted left and right relatively from the first oven wall front image Is1. The movement amount storage array only needs to store at least the left-right movement amount, but the up-down movement amount may also be stored. When storing the horizontal movement amount and the vertical movement amount, the size of the movement amount storage array is the total number of frames×2.

[0101] The inter-frame movement amount calculation unit 130 starts reading the oven wall perspective image Ip (frame) (S303). The frames are read in chronological order (for example, from the pusher side toward the coke discharge side).

[0102] The inter-frame movement amount calculation unit 130 executes frame reading in step S303 and determines whether the frame reading was successful (S305). If the frame cannot be read in step S305, reading up to the final frame has been completed, and the process shown in Fig. 13 ends. If the frame reading is successful in step S305, the inter-frame movement amount calculation unit 130 divides the frame into a left region and a right region using the axis of symmetry C determined in the left / right region calculation process (S307).

[0103] Next, the inter-frame movement amount calculation unit 130 performs perspective transformation on each of the left and right regions of the frame to obtain a normal image of each region (S309). The frame obtained by imaging with the imaging device is a furnace wall perspective image Ip obtained in a state in which the furnace walls 11L and 11R are viewed obliquely as shown on the left side of FIG. 14. In the image processing method according to this embodiment, a furnace wall observation image I is obtained in a state in which the furnace walls 11L and 11R are viewed normal so that the state of the furnace walls 11L and 11R can be confirmed in more detail. Therefore, in step S309, the furnace wall perspective image Ip is perspective transformed to obtain a furnace wall normal image Is. For example, the rectangular right region IpR including the furnace wall 11R in the frame (furnace wall perspective image) Ip on the left side of FIG. 14 becomes a trapezoidal region IsR in the furnace wall normal image Is after perspective transformation as shown on the right side of FIG. 14. The four vertices ABCD of the right region IpR in the frame (oblique oven wall image) Ip correspond to the four vertices abcd of the right region IsR in the normal oven wall image Is.

[0104] Then, the frame movement amount calculation unit 130 corrects the oven wall front view image Is by the filter array generated in step S25 of Fig. 8 in order to make the oven wall front view image Is of the left region and the right region obtained in step S309 clear (S310). The oven wall front view image Is may be corrected according to the above formula (5-1) shown in the first embodiment.

[0105] Next, the frame-to-frame movement amount calculation unit 130 sets a search area for performing template matching processing from the left and right area oven wall front-view images Is corrected in step S310 (S311). The search area is appropriately set by the user to a part in the oven wall front-view image Is where the oven wall can be clearly confirmed, for example as shown in FIG. 15. The size of the search area is preferably set as large as possible to improve the accuracy of the template matching processing. On the other hand, if the entire oven wall front-view image Is is set as the search area, the template matching processing takes time, and the size of the template matching result R obtained by the template matching processing also becomes large. Therefore, it is preferable to set the search area as large as possible and to a part in the oven wall front-view image Is where the oven wall can be clearly confirmed.

[0106] Furthermore, the inter-frame movement amount calculation unit 130 performs preprocessing on the set search area (S313). As the preprocessing, for example, a smoothing process may be performed on the image in the search area. As the preprocessing, for example, an image in which green pixel components are extracted from the image in the search area may be generated. The oven wall shown in the oven wall normal view image Is is generally red, flat, and blurred. When template matching processing is performed with such an image, the template matching result R can be obtained with an acceptable degree of accuracy, but if an image with clearer characteristic parts is used, the template matching result R can be obtained with higher accuracy. In the image of the oven wall, characteristic parts such as the joints of the oven wall appear in the green pixel components. Therefore, by preparing an image in which green pixel components are extracted from the image in the search area as preprocessing, the template matching processing can be performed with higher accuracy.

[0107] Thereafter, the inter-frame movement amount calculation unit 130 judges whether the i-th frame (i is a positive integer, hereinafter also referred to as the "current frame") read in step S303 is the first frame (S315). If the current frame is the first frame, there is no frame to match with the current frame. Therefore, the inter-frame movement amount calculation unit 130 cuts out a template image A2 from the search area obtained by performing preprocessing on the oven wall normal view image Is (S321), and performs the processing from step S303.

[0108] The template image A2 cut out in step S321 is an image based on a frame that has completed the process up to step S319 as the current frame and will be the previous frame in the process from the next step S303. Specifically, the template image A2 is cut out from a preprocessed search area in the oven wall normal image Is obtained by perspective transforming the previous frame. The template image A2 is set separately from the template image A1 set in the left and right area calculation process described above. The template image A2 is set at a predetermined position in the oven wall normal image Is of all frames, for example, at the center of the search area.

[0109] On the other hand, if the current frame read in step S303 is not the first frame, the inter-frame movement amount calculation unit 130 performs template matching processing with the template image A2 based on the i-1th frame (previous frame) read one frame before the current frame in each search area preprocessed on the oven wall front image Is in the left and right areas of the current frame, and calculates the inter-frame movement amount L (S317). The template matching processing is performed by translating the template image A2 of the previous frame cut out in step S321 within the image preprocessed (S313) on the search area set in the oven wall front image Is in step S311. That is, in step S317, the template image A2 of the oven wall front image Is of the previous frame (first oven wall front image Is1) is specified as a matching area in the search area set in the oven wall front image Is of the current frame read next (second oven wall front image Is2).

[0110] In the template matching process, the similarity between the template image A2 of the previous frame and an image of the same size as the template image A2 cut out from the search area of ​​the current frame is calculated. The similarity of the images can be expressed, for example, by the sum of squares of the pixel value differences (Sum of Squared Difference (SSD)) or the sum of absolute values ​​of the pixel value differences (Sum of Absolute Difference (SAD)). The image in the search area with the smallest of these values ​​is the most similar to the template image A2 of the previous frame.

[0111] As described above, the template matching result R obtained by the template matching process is two-dimensional array data that stores numerical values ​​that indicate the degree of similarity between the search target image and the template image.

[0112] When the center of the template image A2 of the previous frame is set to coincide with the center of the search area of ​​the current frame, if the oven wall front view image Is of the current frame is the same image as the oven wall front view image Is of the previous frame (i.e., if there is no misalignment between the first oven wall front view image Is1 and the second oven wall front view image Is2), the part with the highest similarity in the template matching result R will appear in the center of the template matching result R. On the other hand, if the same oven wall part is reflected in different positions in the oven wall front view image Is of the current frame and the oven wall front view image Is of the previous frame, the part with the highest similarity in the template matching result R will appear in a position shifted from the center of the template matching result R.

[0113] Therefore, by finding the relative position from the center position of the template matching result R to the position of the array element with the highest similarity, it is possible to find the inter-frame movement amount L by which the oven wall front-view image Is of the current frame has moved from the oven wall front-view image Is of the previous frame. This makes it possible to know that the part corresponding to the template image A2 in the previous frame has moved by the inter-frame movement amount L in the current frame, for example, as shown in Fig. 15.

[0114] The inter-frame movement amount calculation unit 130 stores the inter-frame movement amount L calculated based on the obtained template matching result R in the movement amount storage array prepared in step S301 (S319). Then, for the next template matching process, a template image A2 is cut out from the preprocessed search area in the oven wall front view image Is of the current frame (S321), and the process from step S303 is carried out. The processes of steps S303 to S321 are repeatedly carried out until there are no more frames to read. When the process shown in FIG. 13 is completed, for example, the movement amount between two frames (inter-frame movement amount L) is obtained in multiple frames acquired in one movement from the pusher side to the coke discharge side.

[0115] (S40: Inter-frame movement amount estimation process) Returning to the explanation of Fig. 8, upon completion of the inter-frame movement amount calculation process (S30), the inter-frame movement amount estimation section 140 carries out an inter-frame movement amount estimation process (S40). Fig. 16 is a flowchart showing an example of the inter-frame movement amount estimation process.

[0116] Inter-frame movement amount estimation unit 140 first sets an estimated movement amount storage array for storing the calculated inter-frame movement amount estimated value E to an initial value (S401). For example, a value of "0" is stored as the initial value. The estimated movement amount storage array is set to a size capable of storing inter-frame movement amount estimated values ​​E for all frames, and an estimated movement amount storage array for the left region and an estimated movement amount storage array for the right region are prepared.

[0117] Furthermore, the inter-frame motion estimation unit 140 calculates an average and a standard deviation from the inter-frame motion amounts L (S403). At this time, the inter-frame motion estimation unit 140 may calculate the average and standard deviation using only the inter-frame motion amounts L that are within a preset reference range among all the inter-frame motion amounts L. By calculating the average and standard deviation without using the inter-frame motion amounts L that are outside the expected values, the effects of noise, etc. can be eliminated.

[0118] 16, the average and standard deviation to be used in the subsequent processing are calculated, but the present invention is not limited to this example. In step S403, the inter-frame movement amount estimation unit 140 calculates a value required to set a likely value of the inter-frame movement amount L (i.e., the inter-frame movement amount estimated value E) instead of the inter-frame movement amount L when the calculated inter-frame movement amount L deviates from an expected value. Therefore, the inter-frame movement amount estimation unit 140 may calculate values ​​other than the average and standard deviation of the inter-frame movement amount L, and may not necessarily calculate the average or standard deviation of the inter-frame movement amount L.

[0119] Then, the inter-frame movement amount estimation unit 140 sets a threshold value for determining whether the inter-frame movement amount L is out of the expected value (S405). The threshold value may be set based on operational experience. For example, the inter-frame movement amount estimation unit 140 may set the value of 2σ (i.e., the average ±2×standard deviation) in the normal distribution of the inter-frame movement amount L as the threshold value.

[0120] Next, the inter-frame movement amount estimation unit 140 sets an index i for counting the number of inter-frame movement amounts L read from the movement amount storage array to 1 (S407), and reads the inter-frame movement amounts L in chronological order (for example, in order from the pusher side toward the coke discharge side). If the index i is equal to or greater than the total number of frames, the inter-frame movement amount estimation unit 140 ends the processing shown in FIG. 16 because it has completed the processing of steps S409 to S417 described below for all inter-frame movement amounts L.

[0121] On the other hand, if index i is less than the total frames, inter-frame movement amount estimation unit 140 reads the inter-frame movement amount L stored in the i-th position of the movement amount storage array, and determines whether or not the i-th inter-frame movement amount L is within the threshold range set in step S405 (S411). If it is determined in step S411 that the i-th inter-frame movement amount L is within the threshold range, the inter-frame movement amount L calculated by inter-frame movement amount calculation unit 130 is considered to be appropriate. Therefore, inter-frame movement amount estimation unit 140 stores the inter-frame movement amount L in the i-th position of the estimated movement amount storage array (S413).

[0122] On the other hand, if it is determined in step S411 that the i-th inter-frame movement amount L is outside the threshold range, the inter-frame movement amount L calculated by the inter-frame movement amount calculation unit 130 is considered to be an inappropriate value that is outside of the expected range. Therefore, the inter-frame movement amount estimation unit 140 stores a plausible value as the inter-frame movement amount L in the i-th position of the estimated movement amount storage array (S415). The plausible value may be, for example, a statistical value statistically determined from the inter-frame movement amount L, such as the average value or median value of the inter-frame movement amount L.

[0123] Upon completing the process of step S413 or S415, inter-frame motion amount estimation unit 140 adds 1 to index i (S417) and repeats the process from step S409.

[0124] (S50: Furnace wall observation image generation processing) Returning to the explanation of Fig. 8, when the inter-frame movement amount estimation process (S40) is completed, the oven wall observation image generating section 150 performs an oven wall observation image generating process (S50). Fig. 17 is a flowchart showing an example of the oven wall observation image generating process.

[0125] The oven wall observation image generating unit 150 first sets initial values ​​to each element of the oven wall observation image array, which is a two-dimensional array that stores pixel values ​​of the images (connection images F) of each frame that constitute the oven wall observation image I to be generated, and to a translation amount variable that indicates the amount of movement of the connection image F of the current frame from the connection image F of the previous frame (S501). For example, a value of "0" is stored as the initial value for each. Each element of the oven wall observation image array and the translation amount variable are prepared for the left region and the right region.

[0126] Next, the oven wall observation image generating unit 150 sets an index i for counting the number of frames read to 0 (S503), and judges whether the index i is less than the total frames (S505). If the index i is equal to or greater than the total frames, the processes in steps S507 to S519 described below are completed for all frames, and the oven wall observation image I is generated, so the process shown in FIG. 17 is terminated.

[0127] On the other hand, if the index i is less than the total number of frames, the oven wall observation image generating unit 150 reads the frame (S507) and adds the value of the corresponding estimated movement amount storage array to the translation amount variable (S509). Then, the oven wall observation image generating unit 150 divides the frame into left and right regions using the symmetry axis C (S511) as performed in the inter-frame movement amount calculation process, and performs perspective transformation on each of the left and right regions of the frame to obtain normal images of each region (S513). Furthermore, the oven wall observation image generating unit 150 corrects the normal images of the left and right regions of the frame by the filter array generated in step S25 of FIG. 8 to sharpen the images (S514). The correction of the normal images may be performed according to the above formula (5-1) shown in the first embodiment.

[0128] The oven wall observation image generating unit 150 cuts out a linking image F constituting the oven wall observation image I from the corrected oven wall front-view image Is (S515). For example, as shown on the left side of FIG. 18, the linking image F is cut out from the oven wall front-view image Is. The area of ​​the linking image F is set in advance to an area in the oven wall front-view image Is where the oven wall can be clearly confirmed. The position and size of the linking image F cut out from the oven wall front-view image Is are the same for each frame.

[0129] Thereafter, the oven wall observation image generating unit 150 moves the cut-out connecting image F along the oven length direction of the oven wall appearing in the oven wall observation image I by the value of the translation amount variable, and stores it in the corresponding part of the oven wall observation image array (S517). For example, as shown on the right side of Fig. 18, the connecting image F is connected to the connecting image F of the previous frame at the corresponding position in the oven length direction of the oven wall appearing in the oven wall observation image I.

[0130] After completing the process of step S517, the oven wall observation image generating unit 150 adds 1 to the index i (S519) and repeats the process from step S505. In this manner, the connection images F are connected, and one oven wall observation image I is generated.

[0131] The image processing device 100 according to the second embodiment and the image processing method using the image processing device have been described above. According to this embodiment, a furnace wall normal time representative image consisting of representative values ​​of pixel values ​​at the same pixel position is generated from a plurality of furnace wall normal images obtained by converting at least a part of each of a plurality of oblique images of the furnace wall sequentially captured by an imaging device moving inside the furnace into normal images, and a filter array is generated for correcting the change in pixel value due to disturbance when the imaging device captures the image based on each pixel value of the furnace wall normal time representative image and the pixel value (e.g., maximum luminance value) of the healthy pixel position that is most likely to be free of disturbance when the imaging device 31, 33 captures the image, and the filter array is used to correct the furnace wall normal image, thereby generating a corrected furnace wall normal image. This makes it possible to obtain a clear and smooth furnace wall observation image that shows the entire furnace wall without using position information of the imaging device that captures the furnace wall.

[0132] [3. Modification (correction of filter arrangement)] When the inside of the furnace is photographed by the imaging device under normal operating conditions, the overall brightness of the photographed image hardly changes with each coke push. However, for example, due to a malfunction such as a decrease in gas flow rate due to damage to the combustion gas inlet caused by refractory deterioration, the temperature inside the coke chamber becomes lower than usual, and the amount of light due to thermal radiation from the furnace wall decreases. In such a situation, the exposure adjustment range of the imaging device may be exceeded, and the entire photographed image may become dark. At this time, it is possible that the proportional relationship or additivity between the dirt attached to the observation window of the imaging device and the amount of light may no longer hold. For example, as a result of correcting the image by the filter arrangement, the dirty part may be over-corrected, and the dirty part may become whitish.

[0133] Therefore, in order to set the filter array appropriately even under such conditions, for example, the shooting conditions may be clustered in advance using the maximum brightness value and the variance value of the brightness value of the time representative image, and the filter array may be corrected according to the class.

[0134] 19 shows an example of a filter correction coefficient determination process for determining a filter correction coefficient for correcting a filter array. In each of the above-described embodiments, such a process can be implemented by adding a filter correction coefficient setting unit that sets in advance a filter correction coefficient for correcting a filter array for each of a plurality of classes into which images are classified, to the image processing device 50, 100.

[0135] In the filter correction coefficient determination process, as shown in FIG. 19, first, the filter correction coefficient setting unit clusters a plurality of time representative value images acquired for each coke extrusion (S40a). The time representative value images may be ones that have already been generated in past processes and have been accumulated. The filter correction coefficient setting unit may cluster the time representative value images, for example, based on the maximum luminance value and the variance value of the luminance value of the time representative value images. The variance value of the time representative value images is obtained from the pixel values ​​in the pixel value conversion area. The variance value is a value that represents the range of luminance change due to dirt, and is an index that measures the overall luminance (distribution of the degree of dirt) from a different perspective from the maximum luminance value. Even if the overall luminance is low, if the range of light reduction due to dirt is narrow, it is considered that proportionality and additivity approximately hold, so it is preferable to cluster according to both the maximum luminance value and the variance value.

[0136] When the time representative value images are clustered in step S40a, the filter correction coefficient setting unit determines a filter correction coefficient for correcting the filter array for each class (S40b). The filter correction coefficient may be determined, for example, using as an index the clarity of an image when the image is corrected using a filter array corrected by the filter correction coefficient set for each class. Specifically, the filter correction coefficient may be temporarily determined, the filter array is corrected and applied to the image, and the user judges the quality of the image, and a filter correction coefficient that provides a good image may be searched for and set. The filter correction coefficient may be, for example, a value that is changed in increments of 0.1 between 0.5 and 1.5.

[0137] The relationship between the class and the filter correction coefficient determined in the filter correction coefficient determination process shown in FIG.

[0138] Fig. 20 is a flowchart showing a process for correcting a filter array in the image processing method according to the first embodiment shown in Fig. 3. In the description of Fig. 20, detailed description of the process performed in the same manner as in Fig. 3 will be omitted.

[0139] (S1: Time representative value image generation process) First, the time representative value image generating unit 51 generates a time representative value image consisting of representative values ​​of pixel values ​​at the same pixel position from a plurality of images of the furnace wall 11 captured sequentially by an imaging device (e.g., the imaging device 31) moving inside the furnace (S1). The time representative value image generating process may be performed in the same manner as step S1 in Fig. 3. The time representative value image generating unit 51 outputs the generated time representative value image to the filter generating unit 53.

[0140] (S3: Filter generation process) Next, the filter generating unit 53 generates a filter array based on each pixel value of the time representative image and the pixel value of a healthy pixel position in the time representative image that is most likely to be free of disturbances caused by the imaging devices 31 and 33 when capturing images (S3). For example, the maximum luminance value of the time representative image can be used as the pixel value of a healthy pixel position in the time representative image that is most likely to be free of disturbances caused by the imaging devices 31 and 33 when capturing images. The filter generating process may be performed in the same manner as step S3 in FIG. 3. The filter generating unit 53 generates a filter array by the process shown in FIG. 5 or FIG. 6, and outputs the generated filter array to the correction processing unit 55.

[0141] (S4: Filter correction coefficient selection process) When the filter array is generated in step S3, the correction processing unit 55 first selects a filter correction coefficient for correcting the image (S4). The filter correction coefficient selection process may be performed, for example, as shown in Fig. 21. Fig. 21 is a flowchart showing an example of the filter correction coefficient selection process.

[0142] First, the correction processing unit 55 calculates the maximum luminance value and the variance value of the luminance value of the time representative image of the image captured by the imaging device (S41). Next, the correction processing unit 55 determines the class to which the time representative image belongs based on the maximum luminance value and the variance value of the luminance value calculated in step S41 (S42). Then, the correction processing unit 55 selects a filter correction coefficient corresponding to the class determined in step S42 based on the relationship between the image class and the filter correction coefficient previously obtained by the process shown in FIG. 19, for example (S43). In this way, it is possible to select a filter correction coefficient for correcting the filter array to be applied to the time representative image.

[0143] (S5: Correction process) Returning to the explanation of FIG. 20, when the filter correction coefficient is selected in step S4, the correction processing unit 55 corrects the image of the inside of the furnace using a correction filter array obtained by correcting the filter array based on the filter correction coefficient corresponding to the selected class (S5).

[0144] For example, when a multiplication filter represented by the above formula (1) is used as the filter array, the value of each pixel of the corrected image is expressed by the following formula (6-1). In the following formula (6-1), a pixel position where f[i,j] is 1 corresponds to a position where the luminance does not need to be corrected by the filter array (a position outside the pixel value conversion area or a position of the maximum luminance value), so p[i,j] is prevented from changing due to the filter correction coefficient. Note that, correction is a filter correction coefficient. When the addition filter represented by the above formula (3) is used as the filter array, the value of each pixel of the corrected image is represented by the following formula (6-2).

[0145]

number

[0146] In this way, by correcting the filter array that corrects the image of the inside of the furnace acquired by the imaging device using a filter correction coefficient appropriate for the image, and correcting the image using the corrected filter array, the image of the inside of the furnace acquired by the imaging device can be made clearer and more appropriate.

[0147] In the above description, the filter array is corrected in the image processing method according to the first embodiment. However, the filter array may be similarly corrected by the filter correction coefficient in the image processing method according to the second embodiment.

[0148] [4. Application Examples] The effect of correcting an image of the inside of a furnace by the image processing method according to the first embodiment will be described with reference to FIGS.

[0149] First, Fig. 22 shows an example of a time-averaged image generated as a time-representative image from a plurality of images of the oven wall 11 captured sequentially by the imaging device by the time-representative image generating unit 51. In Fig. 22, the white parts on the left and right are the oven walls, and the black part at the center is the mouth of the coke oven chamber.

[0150] Fig. 23 shows images before and after correction of the time-averaged image using a filter array generated from the time-averaged image shown in Fig. 22. As shown in Fig. 23, the time-averaged image before correction using the filter array is dark, especially at the boundaries between the upper and lower oven walls, due to dirt on the observation window of the imaging device. When the time-averaged image is corrected using the filter array, the white parts representing the oven walls in the time-averaged image have a uniform brightness.

[0151] Fig. 24 shows images before and after correction of a newly acquired image of the inside of the furnace using a filter array generated from the time-averaged image shown in Fig. 22. As shown in Fig. 24, in the image before correction using the filter array, the boundary part Pb between the upper and lower furnace walls is dark due to the influence of dirt on the observation window of the imaging device. On the other hand, when the image is corrected using the filter array, the white part of the image representing the furnace wall becomes bright and clear overall, and the condition of the furnace wall at the boundary part Pb between the upper and lower furnace walls can be clearly grasped.

[0152] The same effect can be obtained in the image processing method according to the second embodiment. In this way, by using the image processing method according to each embodiment of the present invention, it is possible to clarify the image of the inside of the furnace.

[0153] [5. Hardware configuration] The hardware configuration of the image processing devices 50 and 100 according to this embodiment will be described with reference to Fig. 25. Fig. 25 is a block diagram showing an example of the hardware configuration of an information processing device 900 that functions as the image processing devices 50 and 100 according to this embodiment.

[0154] The information processing device 900 includes a processor (CPU 901 in FIG. 25), a ROM 903, and a RAM 905. The information processing device 900 also includes a bus 907, an input I / F 909, an output I / F 911, a storage device 913, a drive 915, a connection port 917, and a communication device 919.

[0155] The CPU 901 functions as an arithmetic processing device and a control device. The CPU 901 controls all or part of the operations in the information processing device 900 in accordance with various programs recorded in the ROM 903, the RAM 905, the storage device 913, or the removable recording medium 925. The ROM 903 stores programs or arithmetic parameters used by the CPU 901. The RAM 905 temporarily stores programs used by the CPU 901 or parameters that change as appropriate during program execution. These are connected to each other by a bus 907 constituted by an internal bus such as a CPU bus.

[0156] The bus 907 is connected to an external bus such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge.

[0157] The input I / F 909 is an interface that accepts input from an input device 921, which is an operation means operated by a user, such as a mouse, a keyboard, a touch panel, a button, a switch, and a lever. The input I / F 909 is configured as, for example, an input control circuit that generates an input signal based on information input by a user using the input device 921 and outputs the signal to the CPU 901. The input device 921 may be, for example, a remote control device that uses infrared rays or other radio waves, or an external device 927 such as a PDA that supports the operation of the information processing device 900. A user of the information processing device 900 can operate the input device 921 to input various data to the information processing device 900 and instruct the information processing device 900 to perform a processing operation.

[0158] The output I / F 911 is an interface that outputs input information to an output device 923 that can visually or audibly notify a user. The output device 923 may be, for example, a display device such as a CRT display device, a liquid crystal display device, a plasma display device, an EL display device, or a lamp. Alternatively, the output device 923 may be an audio output device such as a speaker and a headphone, a printer, a mobile communication terminal, a facsimile, or the like. The output I / F 911 instructs the output device 923 to output, for example, a processing result obtained in various processes executed by the information processing device 900. Specifically, the output I / F 911 instructs the display device to display the processing result by the information processing device 900 as text or an image. The output I / F 911 also instructs the audio output device to convert an audio signal such as audio data instructed to be played into an analog signal and output it.

[0159] The storage device 913 is one of the storage units of the information processing device 900, and is a device for storing data. The storage device 913 is configured, for example, with a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 913 stores the programs executed by the CPU 901, various data generated by the execution of the programs, various data acquired from the outside, and the like.

[0160] The drive 915 is a reader / writer for a recording medium, and is built in or externally attached to the information processing device 900. The drive 915 reads out information recorded in the attached removable recording medium 925 and outputs it to the RAM 905. The drive 915 can also write information to the attached removable recording medium 925. The removable recording medium 925 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory. Specifically, the removable recording medium 925 may be a CD medium, a DVD medium, a Blu-ray (registered trademark) medium, a CompactFlash (registered trademark) (CF), a flash memory, an SD memory card (Secure Digital memory card), or the like. The removable recording medium 925 may also be, for example, an IC card (Integrated Circuit card) equipped with a non-contact IC chip, or an electronic device, or the like.

[0161] The connection port 917 is a port for directly connecting a device to the information processing device 900. The connection port 917 is, for example, a Universal Serial Bus (USB) port, an IEEE 1394 port, a Small Computer System Interface (SCSI) port, an RS-232C port, etc. The information processing device 900 can directly obtain various data from an external device 927 connected to the connection port 917 and provide various data to the external device 927.

[0162] The communication device 919 is, for example, a communication interface configured with a communication device for connecting to a communication network 929. The communication device 919 is, for example, a wired or wireless LAN (Local Area Network), a communication card for Bluetooth (registered trademark) or WUSB (Wireless USB), or the like. The communication device 919 may also be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various communications. The communication device 919 can transmit and receive signals, for example, between the Internet and other communication devices in accordance with a predetermined protocol such as TCP / IP. The communication network 929 connected to the communication device 919 is configured with a network connected by wire or wirelessly, or the like. For example, the communication network 929 is the Internet, a home LAN, infrared communication, radio wave communication, satellite communication, or the like.

[0163] The above shows an example of the hardware configuration of the information processing device 900. Each of the above-mentioned components may be configured using general-purpose members, or may be configured using hardware specialized for the function of each component. The hardware configuration of the information processing device 900 can be appropriately changed depending on the technical level at the time of implementing this embodiment.

[0164] Although the preferred embodiment of the present invention has been described in detail above with reference to the accompanying drawings, the present invention is not limited to such an example. It is clear that a person having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present invention.

[0165] For example, in the first and second embodiments, when dirt adheres to the observation window of the imaging device, the brightness value at that position decreases, so the position of the pixel having the maximum brightness value in the pixel value conversion region is specified as the pixel position that is definitely free of dirt (step S32a in FIG. 5, S254 in FIG. 12). However, the present invention is not limited to such examples, and a statistical value other than the maximum brightness value may be used to specify a pixel position that is definitely free of dirt, that is, a pixel position that is most likely to be free of disturbance during image capture by the imaging device. For example, focusing on the fact that the higher the average, the larger the variance tends to be, a pixel position that is definitely free of dirt may be specified using a variance value (brightness value time variance) in the time direction of the same pixel position in a moving image (a plurality of images obtained by sequentially capturing images by the imaging device). In this case, the pixel position with the maximum brightness value time variance is the pixel position that is definitely free of dirt. In the above formulas (1) to (4), a filter array can be generated by using the pixel value of the pixel position with the maximum brightness value time variance of the pixel position in question instead of the maximum brightness value of the time average image. [Explanation of symbols]

[0166] 1. Coke oven 10 Carbonization chamber 11, 11L, 11R furnace wall 13 hearth bottom 15 Carbonization chamber mouth 20 Extruded Ram Beam 21 Extrusion Ram 31, 33 Imaging device 50, 100 Image processing device 51 Time representative image generator 53, 160 Filter generation section 55 Correction processing section 110 Image acquisition unit 120 Pre-processing section 130 Inter-frame movement calculation unit 140 Inter-frame movement estimation unit 150 Furnace wall observation image generation unit 300 storage device 310 Captured image storage unit 320 Furnace wall observation image storage unit 500 display device 900 Information processing device 907 Bus 913 Storage Device 915 Drive 917 Connection Port 919 Communication Equipment 921 Input Device 923 Output Device 925 Removable Recording Media 927 External equipment 929 Communication Network A0 partial area A1, A2 template images C Axis of symmetry E Inter-frame motion estimate Ia Time-averaged image Ip Furnace wall perspective image (frame) Right area of ​​IpR furnace wall oblique image Is furnace wall front view image Is1 first furnace wall normal image Is2 2nd furnace wall normal image IsR Right area of ​​the normal image of the furnace wall I Furnace wall observation image F Linking image L Interframe movement amount R Template matching results W Width of time-averaged image

Claims

1. An image processing device that processes an image captured inside a furnace, a time representative image generating unit that generates a time representative image consisting of representative values ​​of pixel values ​​at the same pixel position from a plurality of images of the furnace wall sequentially captured by an imaging device moving inside the furnace; a filter generating unit that generates a filter array for correcting a change in pixel value due to a disturbance when an image is captured by the imaging device, based on each pixel value of the time representative value image and a pixel value of a pixel position in the time representative value image that is most likely to be free of a disturbance when the image is captured by the imaging device; a correction processing unit that corrects an image captured by the imaging device using the filter array; The image processing device includes:

2. 2. The image processing device according to claim 1, wherein the filter generation unit generates a filter array in which, for each pixel value of the time representative value image, a pixel value of a pixel position that has the highest probability of not having a disturbance occurring when the image is captured by the imaging device is divided by the pixel value.

3. 2. The image processing device according to claim 1, wherein the filter generation unit generates a filter array in which, for each pixel value of the time representative image, a pixel value is subtracted from a pixel value of a pixel position that is most likely to be free of disturbance during image capture by the imaging device.

4. a filter correction coefficient setting unit that presets a filter correction coefficient for correcting the filter array for each of a plurality of classes into which the images are classified; The correction processing unit: Identifying a class to which an image captured by the imaging device belongs; 4. The image processing device according to claim 1, further comprising: a filter array correcting unit that corrects the image using the filter array corrected based on a filter correction coefficient corresponding to the specified class.

5. An image processing device that generates a furnace wall observation image for observing the inside of a furnace, a time representative image generating unit that generates a time representative image of the furnace wall in front view from a plurality of furnace wall front view images obtained by converting at least a part of each of the plurality of oblique view images of the furnace wall sequentially captured by an imaging device moving inside the furnace into a front view image, the time representative image being made up of a representative value of pixel values ​​at the same pixel position; a filter generating unit that generates a filter array for correcting a change in pixel value due to disturbance when the image is captured by the imaging device based on each pixel value of the oven wall normal view time representative image and a pixel value of a pixel position in the oven wall normal view time representative image that is most likely to be free of disturbance when the image is captured by the imaging device; a correction processing unit that corrects the oven wall normal image using the filter array to generate a corrected oven wall normal image; The image processing device includes:

6. a frame-to-frame shift amount calculation unit that calculates, by template matching processing, a frame-to-frame shift amount for connecting a first corrected furnace wall front view image and a second corrected furnace wall front view image, which are two corrected furnace wall front view images including an overlapping portion, for the plurality of corrected furnace wall front view images corrected by the correction processing unit; a furnace wall observation image generating unit that generates the furnace wall observation image by connecting the first corrected furnace wall normal image and the second corrected furnace wall normal image based on the inter-frame movement amount; The image processing device according to claim 5 , further comprising:

7. a filter correction coefficient setting unit that pre-sets a filter correction coefficient for correcting the filter array for each of a plurality of classes into which the oven wall normal view images are classified; The correction processing unit: Identifying a class to which the furnace wall normal view image belongs; 7. The image processing device according to claim 5, further comprising: a correction filter array obtained by correcting the filter array based on a filter correction coefficient corresponding to the specified class, and correcting the oven wall normal image to generate the corrected oven wall normal image.

8. An image processing method for processing an image captured inside a furnace, comprising: generating a time representative image consisting of representative values ​​of pixel values ​​at the same pixel position from a plurality of images of the furnace wall sequentially captured by an imaging device moving inside the furnace; generating a filter array for correcting a change in pixel value due to a disturbance when the image is captured by the imaging device, based on each pixel value of the time representative value image and a pixel value of a pixel position in the time representative value image that is most likely to be free of a disturbance when the image is captured by the imaging device; correcting an image captured by the imaging device using the filter array; An image processing method comprising:

9. A program for processing an image of the inside of a furnace, Computer, a time representative image generating unit that generates a time representative image consisting of representative values ​​of pixel values ​​at the same pixel position from a plurality of images of the furnace wall sequentially captured by an imaging device moving inside the furnace; a filter generating unit that generates a filter array for correcting a change in pixel value due to a disturbance when an image is captured by the imaging device, based on each pixel value of the time representative value image and a pixel value of a pixel position in the time representative value image that is most likely to be free of a disturbance when the image is captured by the imaging device; A program that causes the image processing device to function as a correction processing unit that corrects an image captured by the image capturing device using the filter array.

Images