Image processing apparatus, image processing method, and program

Abstract

To provide an image processing device capable of obtaining a clearer image of the interior of a furnace by reducing blur caused by sparks reflected in a captured image.SOLUTION: An image processing device that generates a furnace wall observation image for observing the inside of a furnace comprises: an inter-frame movement amount calculation unit that calculates a deviation amount between first and second furnace wall normal vision images as an inter-frame movement amount; an overlap number calculation unit that calculates an overlap number of connection images cut out from a furnace wall normal vision image at each position in a horizontal direction of the furnace wall observation image based on the inter-frame movement amount; and a furnace wall observation image generation unit that calculates a representative value of pixel values in overlapping portions of the connection images based on the overlap number of the connection images to generate the furnace wall observation image using the representative value, wherein the first and second furnace wall normal vision images are two furnace wall normal vision images including mutually overlapping portions with respect to a plurality of furnace wall normal vision images, and the plurality of furnace wall normal vision images are obtained by converting at least a partial region of each of a plurality of oblique images of a furnace wall sequentially captured by an imaging device moving inside the furnace into a normal vision image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] In a high-temperature furnace chamber such as a carbonization chamber of a coke oven, the furnace wall is made of refractory materials, and accurately grasping the deterioration status of the refractory materials is important for furnace operation management of coke ovens and the like. In particular, the carbonization chamber of a coke oven is continuously operated under severe conditions for a long period of usually 30 years or more. The refractory bricks constituting the carbonization chamber gradually deteriorate due to thermal, chemical, and mechanical factors. For this reason, blockage of coke occurs due to damage to the furnace wall and poor adhesion of carbon, and the resulting blockage causes further damage to the furnace wall, delays in the production schedule, and an increase in the workload for restoration work. Therefore, constantly grasping the deterioration status of the refractory bricks, particularly those constituting the furnace wall, inside the carbonization chamber is extremely important for coke oven operation management.

[0003] As a method for observing the furnace wall of the carbonization chamber of a coke oven, for example, in Patent Document 1, when a camera is inserted inside from the kiln mouth of the coke oven carbonization chamber and the furnace wall condition is observed based on imaging of the inner wall, the field of view of the camera is set obliquely, and the furnace wall is sequentially photographed as a perspective image, and at least almost the entire area in the furnace length direction is photographed with the camera. Based on the imaging position information (position in the furnace length direction) of the perspective image and the image information obtained by converting the perspective image into a front view image, an inner wall observation method for a coke oven carbonization chamber for observing the furnace wall condition inside the carbonization chamber is disclosed.

[0004] In addition, Patent Document 2 discloses a technique for imaging both wall surfaces of a carbonization chamber with separate cameras from directions inclined with respect to their normal directions, and recognizing the types of damage occurring on both wall surfaces by referring to image data provided as perspective images of these wall surfaces. Further, Patent Document 3 discloses a technique for frequency-modulating an output voltage corresponding to the distance to the inner wall surface of a coke oven carbonization chamber measured by a laser distance meter into an audio-band signal by a signal converter and recording it together with an image of the inner wall surface captured by a video camera, thereby associating the image of the inner wall surface with the oven width at the imaging position.

[0005] In this way, by inserting a heat-countermeasure camera into the furnace and observing the furnace wall of the carbonization chamber using the video obtained by photographing the furnace wall, it is possible to observe the furnace wall even at the center of the furnace interior, and it is also possible for the operator to perform the confirmation work safely.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0007] Coke is extruded at a high frequency of once every ten-odd minutes, and cameras and the like are inserted into the high-temperature carbonization chamber. When observing the carbonization chamber every time coke is extruded, in the carbonization chamber, sparks often fly during imaging, and the sparks often appear in the captured image. For this reason, when an imaging image, which is a perspective image, is processed to generate a furnace wall observation image with the furnace wall viewed frontally, the sparks that have entered the field of view of the camera may appear in the furnace wall observation image, and the clarity of the furnace wall observation image may be lost. In order to obtain a clearer furnace wall observation image, it is necessary to reduce the influence of the sparks appearing in the image.

[0008] Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing apparatus, an image processing method, and a program capable of reducing the blurriness caused by the sparks captured in the captured image and obtaining a clearer image of the furnace interior.

Means for Solving the Problems

[0009] In order to solve the above problems, according to an aspect of the present invention, there is provided an image processing apparatus for generating a furnace wall observation image for observing the interior of a furnace, the apparatus including: a frame - to - frame movement amount calculation unit that calculates, as a frame - to - frame movement amount, a deviation amount between a first furnace wall front - view image and a second furnace wall front - view image, which are two furnace wall front - view images including overlapping portions with each other, from a plurality of furnace wall front - view images obtained by converting at least a partial region of each of a plurality of perspective images sequentially captured by an imaging device moving inside the furnace into a front - view image; a duplication number calculation unit that calculates, based on the frame - to - frame movement amount, the number of duplications of connection images cut out from the furnace wall front - view image at each position in the horizontal direction of the furnace wall observation image; and a furnace wall observation image generation unit that calculates a representative value of pixel values in the overlapping portion of the connection images based on the number of duplications of the connection images and generates a furnace wall observation image using the representative value.

[0010] The furnace wall observation image generation unit may calculate a median value as the representative value of pixel values in the overlapping portion of the connection images.

[0011] Alternatively, the furnace wall observation image generation unit may calculate an average value as the representative value of pixel values in the overlapping portion of the connection images.

[0012] The furnace wall observation image generation unit may determine a value calculated as the representative value of pixel values according to the number of duplications of the connection images.

[0013] The frame - to - frame movement amount calculation unit may calculate the frame - to - frame movement amount by template matching processing.

[0014] Here, the image processing apparatus may further include an inter-frame movement amount estimation unit that calculates an estimated value of the inter-frame movement amount for correcting the inter-frame movement amount. At this time, when the calculated inter-frame movement amount is outside the range of a preset threshold value, the inter-frame movement amount estimation value calculated by the inter-frame movement amount estimation unit may be used as the inter-frame movement amount.

[0015] The inter-frame movement amount calculation unit may calculate the inter-frame movement amount based on the extrusion position of the extrusion ram when the perspective image corresponding to the front view image of the furnace wall is captured.

[0016] Also, the inter-frame movement amount calculation unit may calculate the inter-frame movement amount based on the shooting time when the perspective image corresponding to the front view image of the furnace wall is captured.

[0017] The inter-frame movement amount calculation unit may calculate the inter-frame movement amount based on the extrusion speed of the extrusion ram when the perspective image corresponding to the front view image of the furnace wall is captured.

[0018] The inter-frame movement amount calculation unit may calculate the inter-frame movement amount based on the extrusion speed pattern of the extrusion ram in one coke extrusion.

[0019] Also, in order to solve the above problems, according to another aspect of the present invention, there is provided an image processing method for generating a furnace wall observation image for observing the interior of a furnace, the method including: a frame-to-frame movement amount calculation step of calculating, as a frame-to-frame movement amount, a displacement amount between a first furnace wall front view image and a second furnace wall front view image, which are two furnace wall front view images including overlapping portions with each other, for a plurality of furnace wall front view images obtained by converting at least a partial region of each of a plurality of perspective images sequentially captured by an imaging device moving inside the furnace into a front view image; a duplication number calculation step of calculating, based on the frame-to-frame movement amount, the number of overlapping connection images cut out from the furnace wall front view image at each position in the horizontal direction of the furnace wall observation image; and a furnace wall observation image generation step of calculating a representative value of pixel values in the overlapping portion of the connection images based on the number of overlapping connection images and generating a furnace wall observation image using the representative value.

[0020] Furthermore, in order to solve the above problems, according to another aspect of the present invention, there is provided a program for image processing an image captured of the interior of a furnace, the program causing a computer to function as a frame-to-frame movement amount calculation unit that calculates, as a frame-to-frame movement amount, a displacement amount between a first furnace wall front view image and a second furnace wall front view image, which are two furnace wall front view images including overlapping portions with each other, for a plurality of furnace wall front view images obtained by converting at least a partial region of each of a plurality of perspective images sequentially captured by an imaging device moving inside the furnace into a front view image; a duplication number calculation unit that calculates, based on the frame-to-frame movement amount, the number of overlapping connection images cut out from the furnace wall front view image at each position in the horizontal direction of the furnace wall observation image; and a furnace wall observation image generation unit that calculates a representative value of pixel values in the overlapping portion of the connection images based on the number of overlapping connection images and generates a furnace wall observation image using the representative value.

Advantages of the Invention

[0021] As described above, according to the present invention, by calculating the representative value of the pixel values in the overlapping portion of the furnace wall front view images and generating the furnace wall observation image using the representative value, it is possible to reduce the blurriness caused by the flying ash captured in the captured image and obtain a clearer image of the furnace interior.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Embodiments for Carrying Out the Invention

[0023] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0024] [1. Observation of the Carbonization Chamber of the Coke Oven] First, based on FIG. 1, a configuration example of a furnace wall observation device for acquiring an image of the furnace wall of the carbonization chamber of a coke oven will be described. FIG. 1 is a schematic diagram showing a configuration example of a furnace wall observation device according to an embodiment of the present invention.

[0025] The coke oven 1 is a kiln for producing coke. In the upper part of the furnace body of the coke oven 1, carbonization chambers and combustion chambers are alternately arranged, and a regenerator is provided in the lower part. Coke is produced by burning combustion gas and combustion air preheated in the regenerator in the combustion chamber, and carbonizing the coal charged into the carbonization chamber with the generated heat.

[0026] FIG. 1 shows a state of the inside of the carbonization chamber 10 of the coke oven 1 as viewed from the carbonization chamber width direction (X direction). The carbonization chamber 10 is composed of two furnace walls 11 facing each other in the carbonization chamber width direction, and only one furnace wall 11 on the back side of the paper is shown in FIG. 1. The coke produced by carbonizing the coal charged into the carbonization chamber 10 is pushed out to the coke discharge side of the carbonization chamber 10 by moving the extrusion ram 21 provided at one end of the extrusion ram beam 20 in the extrusion direction (Y direction), and is discharged from the carbonization chamber 10.

[0027] On the extrusion ram beam 20 of FIG. 1, two imaging devices 31 and 33 arranged in parallel in the height direction (Z direction) are provided at the center of the width of the coke oven chamber 10. The imaging devices 31 and 33 function as furnace wall observation devices for acquiring furnace wall images of the furnace wall 11 of the coke oven chamber 10. The imaging devices 31 and 33 are provided with a cooling mechanism or a heat-resistant mechanism so as to withstand the high-temperature environment inside the coke oven chamber 10. The imaging devices 31 and 33 shown in FIG. 1 take images of the furnace wall 11 sequentially when the extrusion ram beam 20 is moved from the extruder side to the coke discharge side or when it is moved from the coke discharge side to the extruder side, with the extruder side opposite to the coke discharge side as the imaging direction. Thereby, the inside of the coke oven chamber 10 can be observed every time coke is extruded. The images acquired by the imaging devices 31 and 33 are output to an image processing device described later.

[0028] In addition, although two imaging devices 31 and 33 are installed in FIG. 1, the present invention is not limited to such an example, and the number of imaging devices may be one or three or more. Further, the imaging devices 31 and 33 may be provided on a moving mechanism other than the extrusion ram beam 20 that can move inside the coke oven chamber 10 along the extrusion direction.

[0029] [2. Image Processing Device] The images acquired by the imaging devices 31 and 33 shown in FIG. 1 are output to the image processing device 100 according to the present embodiment. The image processing device 100 according to the present embodiment is a device that generates a furnace wall observation image by connecting a plurality of front-view furnace wall images obtained by perspective transformation of a perspective image of the furnace wall 11 inside the coke oven chamber 10 of the coke oven 1.

[0030] The images acquired by the imaging devices 31 and 33 are not front view images obtained by setting the imaging devices 31 and 33 facing the furnace wall 11 directly, but perspective images obtained by imaging the furnace wall 11 in a perspective state by, for example, installing the imaging devices 31 and 33 with the imaging direction as the extrusion direction. Also, in each perspective image acquired while moving the imaging devices 31 and 33 in the extrusion direction of the carbonization chamber 10, the farther a part is from the position of the imaging devices 31 and 33 at the time of imaging each perspective image, the less clear it becomes, so it is difficult to clearly confirm the entire furnace wall 11 shown in each perspective image. However, the furnace wall 11 in the vicinity of the position in the extrusion direction when each perspective image was taken can be clearly confirmed from each perspective image.

[0031] Therefore, the image processing apparatus 100 according to the present embodiment extracts a portion where the furnace wall can be clearly confirmed from the perspective images captured by the imaging devices 31 and 33, and generates a front view image of the furnace wall by performing a perspective transformation on the extracted partial image (furnace wall perspective image). Then, connection images cut out from a plurality of front view images of the furnace wall are connected to generate a furnace wall observation image representing the entire furnace wall 11.

[0032] Here, in the perspective images captured by the imaging devices 31 and 33, powder of fire may be captured. If an imaging image in which powder of fire is captured is subjected to image processing to generate a furnace wall observation image, the powder of fire will also be captured in the furnace wall observation image, and the sharpness of the furnace wall observation image will be lost. Therefore, in the image processing apparatus 100 according to the present embodiment, a furnace wall observation image is generated using the representative value of the pixel values of the portions (overlapping portions) that image the same position in a plurality of front view images of the furnace wall.

[0033] Figure 2 shows an overview of the generation process of the furnace wall observation image I according to the present embodiment. In the following description, for images such as the furnace wall observation image I, the direction corresponding to the extrusion direction (Y direction) of the furnace wall 11 of the carbonization chamber 10 is defined as the "lateral direction", and the direction corresponding to the height direction (Z direction) of the furnace wall 11 of the carbonization chamber 10 is defined as the "vertical direction". The furnace wall observation image I is composed of connecting a plurality of connection images F. In one coke extrusion, a plurality of captured images are captured. The connection image F is generated by cutting out a front view image of the furnace wall obtained by image processing of the captured image to a predetermined size. In one coke extrusion, since the captured images are continuously captured, some of the connection images F may include portions that capture the same position of the furnace wall 11.

[0034] For example, in Figure 2, the connection images F1 to F7 include a portion (overlapping portion) Po that captures the same position of the furnace wall 11. For example, if sparks are reflected in the overlapping portion Po of the connection images F1 to F7, the pixel value of the overlapping portion Po of the connection image F7 is significantly different from the pixel values of the overlapping portions of the other connection images F1 to F6. As a result, when connecting the connection images in the shooting order, the pixel value of the overlapping portion Po of the connection image F7, which becomes the pixel value of the generated furnace wall observation image I, may become noise on the furnace wall observation image I, and the furnace wall observation image I may become unclear. Therefore, the image processing apparatus 100 according to the present embodiment uses a representative value of the pixel values of the overlapping portion Po of the connection images F1 to F7 instead of using the pixel value of the last connection image F7 among the connection images F1 to F7 having the overlapping portion Po to generate the furnace wall observation image I. Thereby, the influence of the sparks reflected in the image is reduced, and a clearer furnace wall observation image can be obtained.

[0035] Figure 3 shows a configuration example of the image processing apparatus 100 according to the present embodiment. As shown in Figure 3, the image processing apparatus 100 according to the present embodiment includes an image acquisition unit 110, a preprocessing unit 120, an inter-frame movement amount calculation unit 130, an inter-frame movement amount estimation unit 140, a furnace wall observation image generation unit 150, and a duplication number calculation unit 160.

[0036] The image acquisition unit 110 acquires the furnace wall perspective image Ip captured by the imaging devices 31 and 33 from the captured image storage unit 310 of the storage device 300. The captured image storage unit 310 is a storage unit that stores the furnace wall perspective image Ip captured by the imaging devices 31 and 33, and is provided in the storage device 300 configured by, for example, a server or the like. Note that the storage device 300 may be provided in the image processing device 100.

[0037] The imaging devices 31 and 33 continuously image the furnace wall 11 during a single movement from the extruder side to the coke discharge side. Thereby, the imaging devices 31 and 33 obtain a moving image of the imaging target portion of the furnace wall 11, such as the entire furnace wall 11, which is composed of a plurality of furnace wall perspective images Ip (frames) captured in a perspective state of the furnace wall 11. A part of the furnace wall 11 is shown in each furnace wall perspective image Ip, and the part of the furnace wall 11 shown in the furnace wall perspective image Ip includes a part that overlaps with the part of the furnace wall 11 shown in other furnace wall perspective images Ip. Note that it is sufficient that at least two consecutive furnace wall perspective images Ip in the imaging order have an overlapping part, and two discontinuous furnace wall perspective images Ip with one or more frames in between may also have an overlapping part.

[0038] The plurality of furnace wall perspective images Ip captured by the imaging devices 31 and 33 are sequentially recorded in the captured image storage unit 310. Specifically, the imaging devices 31 and 33 and the storage device 300 are communicably connected wirelessly or by wire, and are transmitted from the imaging devices 31 and 33 to the storage device 300. The image acquisition unit 110 outputs the furnace wall perspective image Ip acquired from the captured image storage unit 310 to the preprocessing unit 120, the inter-frame movement amount calculation unit 130, and the furnace wall observation image generation unit 150.

[0039] While moving the imaging devices 31 and 33 from the extruder side to the coke discharge side, the preprocessing unit 120 performs preprocessing necessary to generate the furnace wall front view images Is from the plurality of furnace wall perspective images Ip captured by the imaging devices 31 and 33. When receiving the furnace wall perspective image Ip from the image acquisition unit 110, the preprocessing unit 120 performs a process for appropriately cutting out portions representing the two furnace walls 11 of the carbonization chamber 10 imaged in each of the furnace wall perspective images Ip. Specifically, the preprocessing unit 120 obtains a symmetry axis C (symmetry axis C in FIG. 9) for specifying regions (hereinafter also referred to as the "right region", "left region", or collectively the "left and right regions") of the two furnace walls 11 on the left and right of the carbonization chamber 10 from the furnace wall perspective image Ip. Details of the process for obtaining the symmetry axis C by the preprocessing unit 120 will be described later. The preprocessing unit 120 outputs information for setting the obtained symmetry axis C to the inter-frame movement amount calculation unit 130 and the furnace wall observation image generation unit 150.

[0040] The inter-frame movement amount calculation unit 130 generates a plurality of furnace wall front view images Is using the preprocessing result by the preprocessing unit 120. Then, for the first furnace wall front view image Is1 and the second furnace wall front view image Is2, which are two furnace wall front view images Is including overlapping portions, among the generated plurality of furnace wall front view images Is, the inter-frame movement amount calculation unit 130 calculates the displacement amount between the first furnace wall front view image Is1 and the second furnace wall front view image Is2 as the inter-frame movement amount L. FIG. 4 is a diagram showing the inter-frame movement amount L.

[0041] The inter-frame movement amount calculation unit 130 may calculate the inter-frame movement amount L by, for example, template matching processing. Further, the inter-frame movement amount calculation unit 130 may calculate the inter-frame movement amount L based on, for example, the extrusion position or the shooting time of the extrusion ram 21 when the perspective image Ip of the furnace wall corresponding to the front view image Is of the furnace wall is captured. Furthermore, the inter-frame movement amount calculation unit 130 may calculate the inter-frame movement amount L based on, for example, the extrusion speed of the extrusion ram 21 when the perspective image Ip of the furnace wall corresponding to the front view image Is of the furnace wall is captured, or the extrusion speed pattern of the extrusion ram 21 in one coke extrusion. The method for calculating the inter-frame movement amount L by the inter-frame movement amount calculation unit 130 will be described later.

[0042] The inter-frame movement amount calculation unit 130 outputs the calculated inter-frame movement amount L to the furnace wall observation image generation unit 150 and the duplicate number calculation unit 160. Further, when the inter-frame movement amount calculation unit 130 corrects the inter-frame movement amount L by template matching processing, instead of the inter-frame movement amount L, it outputs the corrected inter-frame movement amount (the inter-frame movement amount estimated value described later) L' calculated by the inter-frame movement amount estimation unit 140.

[0043] The inter-frame movement amount estimation unit 140 calculates an inter-frame movement amount estimated value L' for correcting the inter-frame movement amount L calculated by the inter-frame movement amount calculation unit 130. The inter-frame movement amount L may not be correctly calculated due to the influence of noise such as smoke captured in the captured image. The inter-frame movement amount estimation unit 140 calculates an inter-frame movement amount estimated value as a correction value for correcting to a reasonable value when the inter-frame movement amount L calculated by the inter-frame movement amount calculation unit 130 is outside the threshold range. The inter-frame movement amount estimation unit 140 outputs the calculated inter-frame movement amount estimated value L' to the inter-frame movement amount calculation unit 130. Note that the image processing apparatus 100 may be provided with the inter-frame movement amount estimation unit 140 only when correcting the inter-frame movement amount L by template matching processing.

[0044] The furnace wall observation image generation unit 150 generates a front view image Is of the furnace wall using the perspective image Ip of the furnace wall output from the image acquisition unit 110, cuts out and concatenates the connection images F from the generated plurality of front view images Is of the furnace wall to generate a furnace wall observation image I. The furnace wall observation image generation unit 150 calculates a representative value of the pixel values in the overlapping portion of the connection image F based on the number of overlaps of the connection image F calculated by the overlap number calculation unit 160 described later, and generates the furnace wall observation image I using the representative value. Thereby, when generating the furnace wall observation image I from a plurality of connection images F having an overlapping portion, the influence of a partial region where the fire powder is reflected can be reduced, and a clearer furnace wall observation image I can be obtained. The details of the furnace wall observation image generation process by the furnace wall observation image generation unit 150 will be described later. The furnace wall observation image generation unit 150 may output the generated furnace wall observation image I to a display device 500 communicably connected to the image processing device 100. Alternatively, the furnace wall observation image generation unit 150 may record the generated furnace wall observation image I in the furnace wall observation image storage unit 320 of the storage device 300.

[0045] The overlap number calculation unit 160 calculates the number of overlaps of the connection image F at each position in the horizontal direction (i.e., the extrusion direction (Y direction) of the carbonization chamber) of the furnace wall observation image I based on the inter-frame movement amount L or the corrected inter-frame movement amount L' calculated by the inter-frame movement amount calculation unit 130 or the inter-frame movement amount estimation unit 140. The number of overlaps of the connection image F is used when the furnace wall observation image generation unit 150 generates the furnace wall observation image I. The details of the overlap number calculation process by the overlap number calculation unit 160 will be described later. The overlap number calculation unit 160 outputs the calculated number of overlaps of the front view image Is of the furnace wall to the furnace wall observation image generation unit 150.

[0046] The image processing apparatus 100 according to the present embodiment has been described above. In the image processing apparatus 100 according to the present embodiment, the conversion from the furnace wall perspective image Ip to the furnace wall front view image Is is performed by the inter-frame movement amount calculation unit 130 and the furnace wall observation image generation unit 150, respectively. However, the present invention is not limited to such an example. For example, the furnace wall front view image Is generated by the inter-frame movement amount calculation unit 130 may be passed to the furnace wall observation image generation unit 150, or the furnace wall front view image Is generated by a specific one functional unit may be passed to another functional unit.

[0047] [3. Image Processing Method] Hereinafter, based on FIGS. 5 to 30, the image processing method by the image processing apparatus 100 according to the present embodiment will be described in detail. FIG. 5 shows the overall flow of the image processing method according to the present embodiment, and the details of each process will be described with reference to FIGS. 6 to 30. In the following, the case of obtaining one furnace wall observation image I from a plurality of furnace wall perspective images Ip acquired by one imaging device (for example, imaging device 31) in one movement from the extruder side to the coke discharge side will be described. When images inside the carbonization chamber can be acquired by a plurality of imaging devices, the image processing method described below may be performed for each imaging device.

[0048] [3-1. Furnace Wall Perspective Image Acquisition Process] In the image processing method according to the present embodiment, as shown in FIG. 5, first, the image acquisition unit 110 acquires a plurality of furnace wall perspective images Ip recorded in the captured image storage unit 310 of the storage device 300 (S0). For example, in one movement from the extruder side to the coke discharge side, the imaging device (for example, imaging device 31) acquires a plurality of furnace wall perspective images Ip. The furnace wall perspective image Ip can also be said to be a frame constituting a moving image. The image acquisition unit 110 acquires a plurality of furnace wall perspective images Ip (frames) obtained by continuously imaging the furnace wall 11 by the imaging device (for example, imaging device 31) during one movement of the extrusion ram beam 20 from the captured image storage unit 310. The image acquisition unit 110 outputs the acquired plurality of furnace wall perspective images Ip to the preprocessing unit 120, the inter-frame movement amount calculation unit 130, and the furnace wall observation image generation unit 150.

[0049] [3-2. Pretreatment (Time-Averaged Image Generation Process, Left and Right Region Calculation Process)] Next, the pretreatment unit 120 performs a time-averaged image generation process (S10) and a left and right region calculation process (S20) as pretreatment.

[0050] If the imaging devices 31 and 33 are exactly facing the extrusion direction, the furnace wall 11 is photographed such that the center of the carbonization chamber opening is on the vertical central axis of the furnace wall perspective image Ip. Therefore, the right and left regions of the furnace wall 11 in the furnace wall perspective image Ip can be preset. However, it is difficult hardware-wise to accurately align the imaging devices 31 and 33 with the extrusion direction. Thus, in the present embodiment, by performing the time-averaged image generation process and the left and right region calculation process by the pretreatment unit 120, the deviation is absorbed software-wise even if there is a slight deviation between the vertical central axis of the furnace wall perspective image Ip and the center of the carbonization chamber opening.

[0051] (Time-Averaged Image Generation Process) In the time-averaged image generation process, a time-averaged image used in the left and right region calculation process is generated. The reason for using the time-averaged image instead of a specific furnace wall perspective image Ip to identify the left and right regions in the left and right region calculation process is that the left-right symmetry of the partial image near the carbonization chamber opening of the time-averaged image is higher than that of a specific furnace wall perspective image Ip. There are reflections other than the furnace wall such as sparks and reflections of non-axisymmetric joints in the partial image near the carbonization chamber opening of a specific furnace wall perspective image Ip. For this reason, the partial image near the carbonization chamber opening of a specific furnace wall perspective image Ip may not be suitable for use in estimating the symmetry axis by matching using the left-right reversed image (template image A1) in the left and right region calculation process described later. Therefore, in the present embodiment, a time-averaged image is generated and the left and right region calculation process described later is performed.

[0052] An example of the time-average image generation process is shown in FIG. 6. As shown in FIG. 6, the preprocessing unit 120 first sets to zero a counter that counts each element of the time-average image array, which is a two-dimensional array for storing the pixel values of the time-average image, and the number of frames (S101). For each element of the time-average image array, the sum of the pixel values at each position of the plurality of pixels constituting each frame is stored for the plurality of frames read. Then, the preprocessing unit 120 starts reading (acquiring) the furnace wall perspective image Ip acquired by the image acquisition unit 110 (S103). In step S103, the frames are read in chronological order (for example, in order from the extruder side toward the coke discharge side).

[0053] The preprocessing unit 120 executes frame reading in step S103 and determines whether the frame reading was successful (S105). If the frame reading is successful (S105: YES), each pixel value of the frame is added to the element of the time-average image array corresponding to each pixel (S107). Then, the preprocessing unit 120 adds 1 to the value of the counter (S109), returns to the process of step S103, and reads the next frame. The processes of steps S103 to S109 are repeatedly executed until the reading of all frames is completed.

[0054] On the other hand, if the frame cannot be read in step S105 (S105: NO), since the reading is completed up to the final frame, the sum of the pixel values stored in each element of the time-average image array is divided by the value of the counter to generate a time-average image (S111). The value of the counter corresponds to the number of frames read. In this way, a time-average image that is the average image of the plurality of furnace wall perspective images Ip acquired in one movement from the extruder side to the coke discharge side is acquired.

[0055] (Left and right region calculation process) Next, the preprocessing unit 120 performs left and right region calculation processing. In the left and right region calculation processing, regions representing the two left and right furnace walls 11 of the carbonization chamber 10 are specified from each furnace wall perspective image Ip (frame) (hereinafter, also referred to as the "right region", "left region" of the furnace wall, or collectively as the "left and right regions"). An example of the left and right region calculation processing is shown in FIG. 7.

[0056] First, the preprocessing unit 120 sets and cuts out a partial region A0 that becomes the template image A1 from the time-averaged image obtained by the time-averaged image generation processing in FIG. 6 (S201). In the time-averaged image Ia, for example, as shown in FIG. 8, the two left and right furnace walls 11R and 11L inside the imaged carbonization chamber 10 are shown. Also, the bottom 13 of the carbonization chamber and the carbonization chamber opening 15 on the pusher side of the carbonization chamber 10 may also be shown. The partial region A0 of the time-averaged image Ia is a region including a part of the furnace wall that can be arbitrarily set by the user and has a size smaller than that of the time-averaged image Ia.

[0057] Next, the preprocessing unit 120 horizontally flips the partial region A0 cut out from the time-averaged image Ia in step S201 to obtain a template image A1 as shown on the right side of FIG. 9 (S203). The template image A1 is set to a width obtained by reducing the horizontal width W of the time-averaged image Ia by a length x1 from each of the left end and the right end of the time-averaged image Ia, as shown in the upper part of FIG. 9, for example. The length x1 can be obtained in advance from the correspondence between the horizontal width W of the time-averaged image Ia and the position near the center of the carbonization chamber opening 15. Note that the vertical width of the template image A1 may be set as appropriate because it has little influence on the left and right region calculation processing.

[0058] Then, the preprocessing unit 120 performs template matching processing to identify the image region where the template image A1 matches (S205).

[0059] Template matching processing is an image processing method for obtaining an image area in a search target image that has the highest similarity to a template image. Specifically, when the size of the search target image is N [pixels] × M [pixels] and the size of the template image is n [pixels] × m [pixels] (0 < n ≤ N, 0 < m ≤ M), a two-dimensional array of (N - n + 1) × (M - m + 1) is obtained as the template matching result by the template matching processing. In the (i, j) element (0 ≤ i < N - n + 1, 0 ≤ j < M - m + 1) of the template matching result, a numerical value representing the similarity between the image area (image size: n × m, pixel position (i, j) of the upper left vertex) of the search target image and the template image is stored. If the element in which the highest similarity value is stored in each element of the template matching result is the (ig, jh) element (0 ≤ ig < N - n + 1, 0 ≤ jh < M - m + 1), the image area in the search target image with the pixel position of the upper left vertex being (ig, jh) and the image size being n × m is obtained as the image area with the highest similarity to the template image.

[0060] The preprocessing unit 120 determines a symmetry axis C (S207) such that the left and right regions are symmetric when the time-averaged image Ia is divided in the horizontal direction (i.e., the width direction (X direction) of the coke oven chamber) from the position of the image area in the time-averaged image Ia obtained by the template matching processing that has the highest similarity to the template image A1. For example, as shown in the lower part of FIG. 9, the position of the symmetry axis C is represented by the following formula (1).

[0061] x = (s + W - x1) / 2 ···(1)

[0062] Note that the length s is the length from one end (e.g., the left end) in the horizontal direction of the time-averaged image Ia to one end of the image area where the template image A1 matches when the position of the image area where the template image A1 matches in the time-averaged image Ia is specified. The above formula (1) can be obtained from the relationship x = s + (W - x - x1).

[0063] The symmetry axis C determined in step S207 is used when dividing the furnace wall perspective image Ip (frame) into left and right regions. By obtaining the symmetry axis C in this way, the furnace wall perspective image Ip (frame) can be more accurately divided into a left region and a right region. Also, by horizontally flipping the partial region A0 cut out from the time-averaged image Ia to obtain the template image A1 and performing template matching processing, the symmetry axis C can be estimated with a simple algorithm. The preprocessing unit 120 outputs the position x of the estimated symmetry axis C to the inter-frame movement amount calculation unit 130 and the furnace wall observation image generation unit 150.

[0064] [3-3. Inter-frame Movement Amount Calculation Processing] Returning to the description of FIG. 5, after finishing the time-averaged image generation processing (S10) and the left and right region calculation processing (S20), the inter-frame movement amount calculation unit 130 performs inter-frame movement amount calculation processing to calculate the inter-frame movement amount L (S30). The inter-frame movement amount calculation processing can be performed by various methods. Hereinafter, as examples of the inter-frame movement amount calculation processing, (1) a method using template matching processing, (2) a method based on the extrusion position of the extrusion ram, (3) a method based on the shooting time, (4) a method based on the extrusion speed of the extrusion ram, and (5) a method based on the extrusion speed pattern of the extrusion ram will be described.

[0065] (1) Method Using Template Matching Processing First, based on FIGS. 10 to 14, the inter-frame movement amount calculation processing using template matching processing will be described. FIG. 10 is a flowchart showing an example of the inter-frame movement amount calculation processing using template matching processing. FIG. 11 is an explanatory diagram for explaining the processing of perspective-transforming the furnace wall perspective image into a furnace wall front view image. FIG. 12 is an explanatory diagram showing the relationship between the search region in the current frame and the template image A2. FIG. 13 is a diagram showing the position of the template image A2 in FIG. 12 separately for the previous frame and the current frame. FIG. 14 is a flowchart showing an example of the inter-frame movement amount estimation processing.

[0066] In this method, after obtaining the inter-frame movement amount L by the inter-frame movement amount calculation process using the template matching process shown in FIG. 10, the obtained inter-frame movement amount L is corrected to a plausible value by the inter-frame movement amount estimation process shown in FIG. 14, thereby obtaining the inter-frame movement amount L' used in subsequent processes. Hereinafter, the inter-frame movement amount calculation process and the inter-frame movement amount estimation process using the template matching process will be described.

[0067] (1-1) Inter-frame movement amount calculation process using template matching process First, the inter-frame movement amount L is obtained by the inter-frame movement amount calculation process using the template matching process shown in FIG. 10. As shown in FIG. 10, the inter-frame movement amount calculation unit 130 first sets an initial value in the movement amount storage array, which is a two-dimensional array for storing the calculated inter-frame movement amount L (S301a). As the initial value, for example, a value of "0" is stored. The movement amount storage array is prepared for the left region and the right region obtained by dividing the furnace wall perspective image Ip.

[0068] As described above, the inter-frame movement amount L is the displacement amount between the first furnace wall front view image Is1 and the second furnace wall front view image Is2. That is, as shown in FIG. 4, the inter-frame movement amount L is the amount by which the second furnace wall front view image Is2 is moved from the state where the first furnace wall front view image Is1 and the second furnace wall front view image Is2 having the same image size are overlapped until the portions of the furnace wall that are duplicated in these furnace wall front view images Is1 and Is2 are exactly overlapped. Since the inter-frame movement amount L represents the displacement between the two furnace wall front view images Is1 and Is2, it can be represented by the number of pixels. Since the left-right (horizontal) direction of the furnace wall front view image Is coincides with the extrusion direction of the carbonization chamber, the inter-frame movement amount L is represented by the number of pixels by which the second furnace wall front view image Is2 is relatively shifted left and right from the first furnace wall front view image Is1. It is sufficient to store at least the movement amount in the left-right (horizontal) direction in the movement amount storage array, but the movement amount in the up-down (vertical) direction may also be stored. When storing the movement amount in the left-right direction and the movement amount in the up-down direction, the size of the movement amount storage array is the total number of frames × 2.

[0069] After setting the initial value, the inter-frame movement amount calculation unit 130 starts reading the furnace wall perspective image Ip (frame) (S303a). The frames are read in chronological order (for example, in order from the extruder side toward the coke discharge side).

[0070] The inter-frame movement amount calculation unit 130 executes reading of the frame in step S303a, and determines whether the reading of the frame was successful (S305a). If the frame cannot be read in step S305a (S305a: NO), since the reading is completed up to the final frame, the process shown in FIG. 10 is terminated. When the frame reading is successful in step S305a (S305a: YES), the inter-frame movement amount calculation unit 130 divides the frame into a left region and a right region using the symmetry axis C determined in the left and right region calculation process (S307a).

[0071] Next, the inter-frame movement amount calculation unit 130 performs perspective transformation on each of the left region and the right region of the frame to obtain a front view image Is of the furnace wall for each region (S309a). The frame obtained by imaging with the imaging device is the furnace wall perspective image Ip obtained in a state of obliquely viewing the furnace walls 11L and 11R as shown on the left side of FIG. 11. In the image processing method according to the present embodiment, in order to more detailedly confirm the states of the furnace walls 11L and 11R, a furnace wall observation image I in a state of directly viewing the furnace walls 11L and 11R is obtained. Therefore, in step S309a, the furnace wall perspective image Ip is perspective-transformed into the furnace wall front view image Is. For example, in the rectangular right region IpR including the furnace wall 11R in the frame (furnace wall perspective image) Ip on the left side of FIG. 11, in the furnace wall front view image Is after perspective transformation as shown on the right side of FIG. 11, it becomes a trapezoidal right region IsR. The four vertices ABCD of the right region IpR in the frame (furnace wall perspective image) Ip correspond to the four vertices abcd of the right region IsR in the furnace wall front view image Is.

[0072] Then, the inter-frame movement amount calculation unit 130 sets a search area for performing template matching processing from the front view images Is of the furnace walls in the left region and the right region (S311a). The search area is appropriately set by the user in a portion where the furnace wall can be clearly confirmed in the front view image Is of the furnace wall, for example, as shown in FIG. 12. In order to improve the accuracy of the template matching processing, it is preferable to set the size of the search area as large as possible. On the other hand, if the entire front view image Is of the furnace wall is used as the search area, the template matching processing takes time, and the size of the template matching result obtained by the template matching processing also becomes large. Therefore, it is preferable to set the search area in a portion where the furnace wall can be clearly confirmed in the front view image Is of the furnace wall while making the size as large as possible.

[0073] Furthermore, the inter-frame movement amount calculation unit 130 performs preprocessing on the set search area (S313a). As the preprocessing, for example, a smoothing process may be performed on the image within the search area. Also, as the preprocessing, for example, an image obtained by extracting the green pixel component from the image within the search area may be generated. The furnace wall shown in the front view image Is of the furnace wall is generally red and is uniformly blurred. When the template matching processing is performed on such an image, a template matching result can be obtained with an acceptable degree of accuracy. However, if an image with more distinct feature portions is used, a template matching result can be obtained with higher accuracy. In the image of the furnace wall, feature portions such as joints of the furnace wall appear in the green pixel component. Therefore, by preparing an image obtained by extracting the green pixel component from the image within the search area as the preprocessing, the template matching processing can be performed more accurately.

[0074] Thereafter, the inter-frame movement amount calculation unit 130 determines whether the i-th frame read in step S303a (i is a positive integer; hereinafter also referred to as the "current frame") is the first frame (S315a). If the current frame is the first frame (S315a: YES), there is no frame to be matched with the current frame. Therefore, the inter-frame movement amount calculation unit 130 cuts out the template image A2 from the search area obtained by preprocessing the front view image Is of the furnace wall (S321a) and performs the processing from step S303a.

[0075] Note that as the template image A2, a rectangular area smaller than the search area S is cut out with the position of the center T of the search area S set in the front view image Is of the furnace wall, which is the current frame, as the center point. Also, the template image A2 is set separately from the template image A1 set in the above-described left and right area calculation process. T of the search area S T is cut out. Also, the template image A2 is set separately from the template image A1 set in the above-described left and right area calculation process.

[0076] On the other hand, if the current frame read in step S303a is not the first frame (S315a: NO), the inter-frame movement amount calculation unit 130 performs template matching processing between the template image A2 (the template image A2 cut out in S321a above) based on the front view image Is1 of the furnace wall of the (i - 1)-th frame (previous frame) read one frame before the current frame and the search area S obtained by preprocessing each of the left and right areas of the current frame in the front view image Is2 of the furnace wall, and calculates the inter-frame movement amount L (S317a). T Here, based on FIG. 13, the template matching process will be described. FIG. 13 is a diagram showing the front view image of the furnace wall in FIG. 12 divided into the previous frame and the current frame respectively. In FIG. 13, it is assumed that the front view image Is1 of the furnace wall is the front view image of the previous frame and the front view image Is2 of the furnace wall is the front view image of the current frame.

[0077] Here, based on FIG. 13, the template matching process will be described. FIG. 13 is a diagram showing the front view image of the furnace wall in FIG. 12 divided into the previous frame and the current frame respectively. In FIG. 13, it is assumed that the front view image Is1 of the furnace wall is the front view image of the previous frame and the front view image Is2 of the furnace wall is the front view image of the current frame.

[0078] As shown in the upper right figure of FIG. 13, in the template matching process, in the image obtained by performing preprocessing (S313a) on the search area S set in the front view image Is2 of the furnace wall in the current frame at step S311a, the template image A2 of the front view image Is1 of the furnace wall in the previous frame cut out at step S321a (the template image A2 in the lower right figure of FIG. 13) is translated. That is, at step S317a, the template image A2 of the front view image Is (the first front view image Is1) of the furnace wall in the previous frame is used to identify the matching area A3 within the search area S set in the front view image Is (the second front view image Is2) of the furnace wall in the next read current frame. T In the template matching process, the similarity between the template image A2 of the front view image Is1 of the furnace wall in the previous frame and the image (area A3) of the same size as the template image A2, which is cut out by translating the template image A2 of the front view image Is1 of the furnace wall in the previous frame within the search area S of the front view image Is2 of the furnace wall in the current frame, is calculated. The similarity of the images can be represented by, for example, the sum of squared differences (SSD) of the pixel values or the sum of absolute differences (SAD) of the pixel values. The image in the search area where these values are the smallest will be the most similar to the template image A2 of the previous frame. T The template matching result obtained by the template matching process is, as described above, two-dimensional array data storing numerical values representing the similarity between the search target image and the template image.

[0079] In the template matching process, the similarity between the template image A2 of the front view image Is1 of the furnace wall in the previous frame and the search area S of the front view image Is2 of the furnace wall in the current frame is calculated. T In the search area S of the front view image Is2 of the furnace wall in the current frame, the template image A2 of the front view image Is1 of the furnace wall in the previous frame is translated and cut out. The similarity between the template image A2 and the image (area A3) of the same size as the template image A2 is calculated. The similarity of the images can be represented by, for example, the sum of squared differences (SSD) of the pixel values or the sum of absolute differences (SAD) of the pixel values. The image in the search area where these values are the smallest will be the most similar to the template image A2 of the previous frame.

[0080] The template matching result obtained by the template matching process is, as described above, two-dimensional array data storing numerical values representing the similarity between the search target image and the template image.

[0081] When the center of the template image A2 of the front view image Is1 of the furnace wall in the previous frame is set to coincide with the center of the search area of the front view image Is2 of the furnace wall in the current frame, if the front view image Is2 of the furnace wall in the current frame is the same image as the front view image Is1 of the furnace wall in the previous frame (that is, if there is no deviation between the first front view image Is1 of the furnace wall and the second front view image Is2 of the furnace wall), in the template matching result, the part with the highest similarity appears at the center of the template matching result. On the other hand, if the same furnace wall part is reflected at different positions in the front view image Is2 of the furnace wall in the current frame and the front view image Is1 of the furnace wall in the previous frame, the part with a high similarity in the template matching result appears at a position deviated from the center of the template matching result, as shown in the upper right figure of Fig. 13.

[0082] Therefore, in step S317a, the inter-frame movement amount calculation unit 130 calculates the relative position from the center position of the template matching result (position T in the lower right figure of Fig. 13) to the position of the array element with the highest similarity (position T' in the upper right figure of Fig. 13), that is, the horizontal deviation amount of the overlapping part (template image A2, area A3) between the first front view image Is1 of the furnace wall and the second front view image Is2 of the furnace wall. The relative position (the horizontal distance between position T and position T') obtained here corresponds to the inter-frame movement amount L by which the front view image Is2 of the furnace wall in the current frame has moved from the front view image Is1 of the furnace wall in the previous frame.

[0083] The inter-frame movement amount calculation unit 130 stores the inter-frame movement amount L calculated based on the obtained template matching result in the movement amount storage array prepared in step S301a (S319a). Then, for the next template matching process, the template image A2 is cut out from the pre-processed search area in the front view image Is2 of the furnace wall in the current frame (S321a), and the process from step S303a is carried out. Note that the template image A2 cut out here is the search area S T centered on the position of the center T of TA rectangular area of a smaller size is newly cut out. The processes of steps S303a to S321a are repeatedly executed until there are no more frames to read. When the process shown in FIG. 10 is completed, for example, in a plurality of frames obtained in one movement from the extruder side to the coke discharge side, the movement amount between two frames (inter-frame movement amount L) can be obtained.

[0084] (1-2) Inter-frame movement amount estimation process Next, the inter-frame movement amount estimation unit 140 corrects the inter-frame movement amount L obtained by the inter-frame movement amount calculation process using the template matching process shown in FIG. 10 to a plausible value by the inter-frame movement amount estimation process shown in FIG. 14.

[0085] The inter-frame movement amount estimation unit 140 first sets an initial value in a determined movement amount storage array for storing the estimated value of the inter-frame movement amount to be calculated (S331a). As the initial value, for example, a value of "0" is stored. The determined movement amount storage array is set to a size capable of storing the estimated values of the inter-frame movement amounts for the total number of frames read in step S105, and is prepared for the left region and the right region. The estimated value of the inter-frame movement amount stored in the determined movement amount storage array by executing the process shown in FIG. 14 becomes the corrected inter-frame movement amount L' used in subsequent processes.

[0086] Also, the inter-frame movement amount estimation unit 140 calculates the average and standard deviation from the plurality of inter-frame movement amounts L (S333a). At this time, the inter-frame movement amount estimation unit 140 may calculate the average and standard deviation using only the inter-frame movement amounts L within a preset reference range among all the inter-frame movement amounts L. By calculating the average and standard deviation without using the inter-frame movement amounts L that deviate from the assumed values, the influence of noise and the like can be excluded.

[0087] Note that, in FIG. 14, the average and standard deviation used in subsequent processing are calculated, but the present invention is not limited to such an example. In step S333a, when the calculated inter-frame movement amount L deviates from an assumed value, the inter-frame movement amount estimation unit 140 calculates a value necessary for setting a likely value (i.e., an inter-frame movement amount estimated value) L' of the inter-frame movement amount L instead of the inter-frame movement amount L. Therefore, the inter-frame movement amount estimation unit 140 may calculate a value other than the average and standard deviation of the inter-frame movement amount L, and does not necessarily have to calculate the average or standard deviation of the inter-frame movement amount L.

[0088] Then, the inter-frame movement amount estimation unit 140 sets a threshold value for determining whether the inter-frame movement amount L deviates from an assumed value (S335a). The threshold value may be set based on operational experience. For example, the inter-frame movement amount estimation unit 140 may use the value of 2σ (i.e., average ± 2 × standard deviation) in the normal distribution of the inter-frame movement amount L as the threshold value.

[0089] Next, the inter-frame movement amount estimation unit 140 sets an index i for counting the number of reads of the inter-frame movement amount L stored in the movement amount storage array to 1 (S337a), and reads the inter-frame movement amount L in chronological order (e.g., in order from the extruder side to the coke discharge side). If the index i is greater than or equal to the total number of frames (S339a: NO), the inter-frame movement amount estimation unit 140 has completed the processing of steps S339a to S347a described below for all the inter-frame movement amounts L, and thus ends the processing shown in FIG. 14.

[0090] On the other hand, when the index i is less than the total number of frames (S339a: YES), the inter-frame movement amount estimation unit 140 reads the inter-frame movement amount L stored in the i-th position of the movement amount storage array, and determines whether the i-th inter-frame movement amount L is within the range of the threshold value set in step S335a (S341a). When it is determined in step S341a that the i-th inter-frame movement amount L is within the range of the threshold value (S341a: YES), the inter-frame movement amount L calculated by the inter-frame movement amount calculation unit 130 is considered appropriate. Therefore, the inter-frame movement amount estimation unit 140 stores the inter-frame movement amount L in the i-th position of the determined movement amount storage array (S343a).

[0091] On the other hand, when it is determined in step S341a that the i-th inter-frame movement amount L is outside the range of the threshold value (S341a: NO), the inter-frame movement amount L calculated by the inter-frame movement amount calculation unit 130 is considered an inappropriate value deviated from the assumption. Therefore, the inter-frame movement amount estimation unit 140 stores a plausible value as the inter-frame movement amount L in the i-th position of the determined movement amount storage array (S345a). Examples of the plausible value include statistical values statistically obtained from the inter-frame movement amount L, such as the average value or median value of the inter-frame movement amount L.

[0092] After finishing the process of step S343a or S345a, the inter-frame movement amount estimation unit 140 adds 1 to the index i (S347a), and repeats the process from step S339a.

[0093] As described above, the method for obtaining the inter-frame movement amount using the template matching process has been explained. The value stored in the determined movement amount storage array by executing the process shown in FIG. 14 is used as the inter-frame movement amount (i.e., the inter-frame movement amount estimated value) used in the subsequent process.

[0094] (2) Method based on the extrusion position of the extrusion ram Next, based on FIG. 15, the calculation process of the inter-frame movement amount using the extrusion position of the extrusion ram will be described. FIG. 15 is a flowchart showing an example of the calculation process of the inter-frame movement amount using the extrusion position of the extrusion ram. In such a method, the inter-frame movement amount calculation unit 130 calculates the inter-frame movement amount L based on the extrusion position of the extrusion ram 21 when the furnace wall perspective image Ip corresponding to the furnace wall front view image Is is captured.

[0095] The extrusion position of the extrusion ram 21 can be calculated, for example, based on the rotation speed of the motor that drives the extrusion ram beam 20. For example, the rotation speed of the motor can be detected by providing a rotary encoder to the motor. The extrusion position of the extrusion ram 21 can be obtained by integrating the cumulative value of the rotation speed of the motor and the movement amount of the extrusion ram beam 20 per rotation. By associating and recording the cumulative value of the rotation speed of the motor when the furnace wall perspective image Ip is captured in the captured image storage unit 310, the extrusion position of the extrusion ram 21 when the furnace wall perspective image Ip is captured can be specified.

[0096] The inter-frame movement amount calculation unit 130 first sets an initial value in the determined movement amount storage array for storing the calculated inter-frame movement amount, and sets an initial value in the position variable representing the extrusion position when the immediately preceding frame was captured (S301b). As the initial value, for example, a value of "0" is stored in the determined movement amount storage array. The determined movement amount storage array is set to a size capable of storing the inter-frame movement amounts for the total number of frames read in step S105, and is prepared for the left region and the right region. Also for the position variable, as the initial value, for example, a value of "0" is stored.

[0097] Next, the inter-frame movement amount calculation unit 130 sets an index i for counting the number of frames read to 1 (S303b), and reads the frames in chronological order (for example, in order from the extruder side toward the coke discharge side). Then, the inter-frame movement amount calculation unit 130 determines whether the index i is smaller than the total number of frames (S305b). If the index i is greater than or equal to the total number of frames (S305b: YES), since the processes of steps S307b to S311b described below have been completed for all frames, the process shown in FIG. 15 is terminated.

[0098] On the other hand, when the index i is less than the total number of frames (S305b: NO), the inter-frame movement amount calculation unit 130 obtains the inter-frame movement amount L of the i-th frame based on the extrusion position of the extrusion ram 21 (S307b). The inter-frame movement amount L of the i-th frame can be obtained, for example, using the following formula (2).

[0099] Inter-frame movement amount L of the i-th frame =(Extrusion position at the time of photographing the i-th frame - position variable) × pixel conversion constant ···(2)

[0100] The inter-frame movement amount calculation unit 130 calculates the distance between the (i - 1)-th frame and the i-th frame by subtracting the value stored in the current position variable from the extrusion position at the time of photographing the i-th frame. Then, the inter-frame movement amount calculation unit 130 obtains the inter-frame movement amount L of the i-th frame represented in pixels by multiplying the distance between the (i - 1)-th frame and the i-th frame by the pixel conversion constant. The pixel conversion constant for converting the unit from position to pixel is a value specific to the coke oven, which is set in advance. For example, using the furnace wall perspective image obtained in the past operation, the pixel conversion constant may be changed to repeatedly create the furnace wall observation image, and the pixel conversion constant when a good furnace wall observation image is obtained may be adopted.

[0101] When the inter-frame movement amount calculation unit 130 calculates the inter-frame movement amount L of the i-th frame using the above formula (2), it stores the inter-frame movement amount L in the i-th position of the determined movement amount storage array.

[0102] Thereafter, the inter-frame movement amount calculation unit 130 substitutes the extrusion position at the time of shooting the i-th frame into the position variable (S309b), adds 1 to the index i (S311b), and repeatedly performs the process from step S305b. The method of obtaining the inter-frame movement amount using the extrusion position of the extrusion ram has been described above.

[0103] (3) Method based on shooting time Next, based on FIG. 16, the inter-frame movement amount calculation process using the shooting time will be described. FIG. 16 is a flowchart showing an example of the inter-frame movement amount calculation process using the shooting time. In such a method, the inter-frame movement amount calculation unit 130 calculates the inter-frame movement amount L based on the shooting time when the furnace wall perspective image Ip corresponding to the furnace wall front view image Is is captured.

[0104] Here, it is assumed that the extrusion speed is constant. Measurement of the extrusion position is not required. In this case, the shooting time when the furnace wall perspective image Ip is captured can be obtained, for example, by multiplying the time from the start of extrusion to the end of extrusion by the value obtained by dividing the frame number (i.e., the index i) by the total number of frames. The time from the start of extrusion to the end of extrusion can be obtained, for example, from the time of the video in which the imaging devices 31 and 33 continuously capture the furnace wall 11 in one movement from the extruder side to the coke discharge side. When the shooting time when the furnace wall perspective image Ip is captured is recorded in the captured image storage unit 310 in association with the furnace wall perspective image Ip, the shooting time recorded in the captured image storage unit 310 may be used.

[0105] The inter-frame movement amount calculation unit 130 first sets an initial value in a determined movement amount storage array for storing the calculated inter-frame movement amount (S301c). For example, a value of "0" is stored in the determined movement amount storage array as the initial value. The determined movement amount storage array is set to a size capable of storing the inter-frame movement amounts for the total number of frames read in step S105, and is prepared for the left region and the right region.

[0106] Next, the inter-frame movement amount calculation unit 130 sets an index i for counting the number of frames read to 1 (S303c), and reads the frames in chronological order (for example, in order from the extruder side toward the coke discharge side). Then, the inter-frame movement amount calculation unit 130 determines whether the index i is smaller than the total number of frames (S305c). If the index i is greater than or equal to the total number of frames (S305c: YES), since the processes of steps S307c and S309c described below have been completed for all frames, the process shown in FIG. 16 is terminated.

[0107] On the other hand, when the index i is less than the total number of frames (S305c: NO), the inter-frame movement amount calculation unit 130 obtains the inter-frame movement amount L of the i-th frame based on the shooting time (S307c). The inter-frame movement amount L of the i-th frame can be obtained, for example, using the following formula (3).

[0108] Inter-frame movement amount L of the i-th frame = Shooting time of the i-th frame × Pixel conversion constant =(1 / Total number of frames) × Time from start of extrusion to end of extrusion × Pixel conversion constant ···(3)

[0109] The inter-frame movement amount calculation unit 130 obtains the inter-frame movement amount L of the i-th frame by multiplying the shooting time of the i-th frame by a pixel conversion constant that converts the shooting time into pixels, as shown in the first equation of Equation (3). The pixel conversion constant for converting the shooting time into pixels is a value specific to the coke oven that is set in advance. For example, using the furnace wall perspective image obtained in past operations, the pixel conversion constant may be changed to repeatedly create the furnace wall observation image, and the pixel conversion constant when a good furnace wall observation image is obtained may be adopted. Note that, as shown in the second equation of Equation (3), the inter-frame movement amount calculation unit 130 may obtain the inter-frame movement amount L of the i-th frame by multiplying the time from the start of extrusion to the end of extrusion by the pixel conversion constant that converts the shooting time into pixels and then dividing by the total number of frames.

[0110] When the inter-frame movement amount calculation unit 130 calculates the inter-frame movement amount L of the i-th frame using the above Equation (3), it stores the inter-frame movement amount L in the i-th position of the determined movement amount storage array.

[0111] After that, the inter-frame movement amount calculation unit 130 adds 1 to the index i (S309c) and repeatedly performs the process from step S305c. The method of obtaining the inter-frame movement amount using the shooting time has been described above.

[0112] (4) Method based on the extrusion speed of the extrusion ram Next, based on FIG. 17, the inter-frame movement amount calculation process using the extrusion speed of the extrusion ram will be described. FIG. 17 is a flowchart showing an example of the inter-frame movement amount calculation process using the extrusion speed of the extrusion ram. In such a method, the inter-frame movement amount calculation unit 130 calculates the inter-frame movement amount L based on the extrusion speed of the extrusion ram 21 when the furnace wall perspective image Ip corresponding to the furnace wall front view image Is is captured.

[0113] The extrusion speed of the extrusion ram 21 can be measured, for example, by attaching a speedometer to the extrusion ram beam 20. Measurement of the extrusion position is unnecessary. The extrusion speed obtained by the speedometer is recorded in the captured image storage unit 310 in association with, for example, the captured perspective image Ip of the furnace wall, and is referred to when performing the inter-frame movement amount calculation process shown in FIG. 17.

[0114] The inter-frame movement amount calculation unit 130 first sets an initial value in a determined movement amount storage array for storing the calculated inter-frame movement amount (S301d). In the determined movement amount storage array, a value such as "0" is stored as the initial value. The determined movement amount storage array is set to a size capable of storing the inter-frame movement amounts for the total number of frames read in step S105, and is prepared for the left region and the right region.

[0115] Next, the inter-frame movement amount calculation unit 130 sets an index i for counting the number of frames read to 1 (S303d), and reads the frames in chronological order (for example, in order from the extruder side toward the coke discharge side). Then, the inter-frame movement amount calculation unit 130 determines whether the index i is smaller than the total number of frames (S305d). If the index i is greater than or equal to the total number of frames (S305d: YES), since the processes of steps S307d and S309d described below have been completed for all frames, the process shown in FIG. 17 is terminated.

[0116] On the other hand, when the index i is less than the total number of frames (S305d: NO), the inter-frame movement amount calculation unit 130 obtains the inter-frame movement amount L of the i-th frame based on the extrusion speed of the extrusion ram 21 (S307d). The inter-frame movement amount L of the i-th frame can be obtained, for example, using the following formula (4).

[0117] Inter-frame movement amount L of the i-th frame = Extrusion speed at the time of shooting the i-th frame × Pixel conversion constant ···(4)

[0118] The inter-frame movement amount calculation unit 130 obtains the inter-frame movement amount Li of the i-th frame by multiplying the extrusion speed at the time of photographing the i-th frame by a pixel conversion constant that converts the extrusion speed into pixels. The pixel conversion constant for converting the extrusion speed into pixels is a value specific to the coke oven, which is set in advance. For example, using a perspective image of the furnace wall obtained in past operations, the pixel conversion constant may be changed to repeatedly create a furnace wall observation image, and the pixel conversion constant when a good furnace wall observation image is obtained may be adopted.

[0119] When the inter-frame movement amount calculation unit 130 calculates the inter-frame movement amount Li of the i-th frame using the above formula (4), the inter-frame movement amount Li is stored in the i-th position of the determined movement amount storage array.

[0120] Thereafter, the inter-frame movement amount calculation unit 130 adds 1 to the index i (S309d) and repeatedly executes the process from step S305d. The method of obtaining the inter-frame movement amount using the extrusion speed has been described above.

[0121] (5) Method based on the extrusion speed pattern of the extrusion ram Next, based on FIGS. 18 and 19, the inter-frame movement amount calculation process using the extrusion speed pattern of the extrusion ram will be described. FIG. 18 is a flowchart showing an example of the inter-frame movement amount calculation process using the extrusion speed pattern of the extrusion ram. FIG. 19 is an explanatory diagram showing an example of the extrusion speed pattern. In such a method, the inter-frame movement amount calculation unit 130 calculates the inter-frame movement amount L based on the extrusion speed pattern of the extrusion ram 21 in one coke extrusion.

[0122] The extrusion speed pattern of the extrusion ram 21 indicates the extrusion speed at each extrusion position from the start to the end of extrusion, as shown in, for example, FIG. 19. In the example of FIG. 19, the extrusion speed increases for a while from the start of extrusion, then becomes almost constant, and then decreases rapidly. Such a change in the extrusion speed at the extrusion position can be considered as a change in the moving distance of the extrusion ram 21. That is, when the extrusion speed is small, the moving distance of the extrusion ram 21 also becomes small, and when the extrusion speed is large, the moving distance of the extrusion ram 21 also becomes large. Therefore, the value obtained by dividing the frame number (i.e., the index i) by the total number of frames is used as the extrusion position parameter λ, and the inter-frame movement amount is calculated from the extrusion speed corresponding to the extrusion position parameter λ in the extrusion speed pattern.

[0123] Note that the extrusion speed pattern is specific to the coke oven. For example, the extrusion speed may be measured and set during the extrusion, or may be set through trial and error based on past operation results. Measurement of the extrusion position is not required.

[0124] The inter-frame movement amount calculation unit 130 first sets an initial value in a determined movement amount storage array for storing the calculated inter-frame movement amount (S301e). For example, a value of "0" is stored in the determined movement amount storage array as the initial value. The determined movement amount storage array is set to a size capable of storing the inter-frame movement amounts for the total number of frames read in step S105, and is prepared for the left region and the right region.

[0125] Next, the inter-frame movement amount calculation unit 130 sets the index i for counting the number of frames read to 1 (S303e), and reads the frames in chronological order (for example, in order from the extruder side toward the coke discharge side). Then, the inter-frame movement amount calculation unit 130 determines whether the index i is smaller than the total number of frames (S305e). If the index i is greater than or equal to the total number of frames (S305e: YES), since the processes of steps S307e to S311e described below have been completed for all the frames, the process shown in FIG. 18 is terminated.

[0126] On the other hand, when the index i is less than the total number of frames (S305e: NO), the inter-frame movement amount calculation unit 130 calculates the extrusion position parameter λ as the value obtained by dividing the frame number (i.e., the index i) by the total number of frames (S307e). Then, the inter-frame movement amount calculation unit 130 obtains the inter-frame movement amount L of the i-th frame based on the extrusion speed pattern (S309e). The inter-frame movement amount L of the i-th frame can be obtained, for example, using the following formula (5).

[0127] Inter-frame movement amount L of the i-th frame = Extrusion speed at the extrusion position parameter λ × Pixel conversion constant ···(5)

[0128] The inter-frame movement amount calculation unit 130 obtains the inter-frame movement amount L of the i-th frame by multiplying the extrusion speed at λ when the value obtained by dividing the frame number (i.e., the index i) by the total number of frames is used as the extrusion position parameter λ by the pixel conversion constant for converting the extrusion speed into pixels. The pixel conversion constant for converting the extrusion speed into pixels is set based on the set extrusion speed pattern. When the inter-frame movement amount calculation unit 130 calculates the inter-frame movement amount L of the i-th frame using the above formula (5), it stores the inter-frame movement amount L in the i-th position of the determined movement amount storage array.

[0129] Thereafter, the inter-frame movement amount calculation unit 130 adds 1 to the index i (S311e) and repeatedly performs the processing from step S305e. The method for obtaining the inter-frame movement amount using the extrusion speed pattern has been described above.

[0130] In this way, the inter-frame movement amount calculation unit 130 performs the inter-frame movement amount calculation process using the method as described above, and calculates the inter-frame movement amount L or the corrected inter-frame movement amount L'.

[0131] [3-4. Frame overlap number calculation process] Returning to the description of FIG. 5, when the inter-frame movement amount calculation process (S30) is completed, the overlapping number calculation unit 160 calculates the number of overlapping frames (S40). Based on FIGS. 20 and 21, the frame overlapping number calculation process will be described. FIG. 20 is a flowchart showing an example of the frame overlapping number calculation process according to the present embodiment. FIG. 21 is an explanatory diagram for explaining the frame overlapping number calculation process. The frame overlapping number calculation process is a process of counting how many connection images F including the overlapping part Po that images the same position of the furnace wall 11 are there, as shown in FIG. 2.

[0132] First, the overlapping number calculation unit 160 obtains the horizontal size of the entire furnace wall observation image I based on the value stored in the determined movement amount storage array (that is, the inter-frame movement amount L or the corrected inter-frame movement amount L' between all consecutive front view images Is of the furnace wall) and the previously determined horizontal size (fixed value) of the connection image (S401). Specifically, the value obtained by adding the horizontal size of the connection image for one frame to the sum of the values stored in the determined movement amount storage array is the horizontal size of the furnace wall observation image I.

[0133] Also, the overlapping number calculation unit 160 sets initial values for the overlapping number storage array for storing the calculated number of overlapping connection images and the parallel movement amount variable j (S403). The overlapping number storage array is set to the same size as the horizontal size of the furnace wall observation image I obtained in step S401, and is prepared for the left region and the right region. For example, a value of "0" is stored in the overlapping number storage array as the initial value. The parallel movement amount variable j is a value (pixel value) indicating the start position of the connection image for each frame in the furnace wall observation image I. The parallel movement amount variable j is also prepared for the left region and the right region.

[0134] Next, the duplicate number calculation unit 160 sets an index i for counting the number of loaded frames to 0 (S405), and reads the frames in chronological order (for example, in order from the extruder side to the coke discharge side). Then, the duplicate number calculation unit 160 determines whether the index i is less than the total number of frames (S407). If the index i is greater than or equal to the total number of frames (S407: YES), since the processing of steps S409 to S419 described later has been completed for all the frames, the processing shown in FIG. 20 is terminated.

[0135] On the other hand, when the index i is less than the total number of frames (S407: NO), the duplicate number calculation unit 160 adds the value stored in the i-th position of the determined movement amount storage array to the parallel movement amount variable j (S409). Then, the duplicate number calculation unit 160 sets an index k representing the horizontal position of the concatenated image to 0 (S411), and determines whether the index k is less than the horizontal size (pixel value) of the concatenated image (S413).

[0136] When the index k is less than the horizontal size of the concatenated image (S413: YES), the duplicate number calculation unit 160 adds the parallel movement amount variable j and the index k to obtain a position j1 in the duplicate number storage array (S415), and adds 1 to the j1-th position of the duplicate number storage array (S417). Then, 1 is added to the index k, and the processing from step S413 is repeatedly performed.

[0137] On the other hand, in step S413, when it is determined that the index j is greater than or equal to the horizontal size of the concatenated image (S413: NO), the duplicate number calculation unit 160 has completed the processing of steps S415 and S417 for the frame (concatenated image), so 1 is added to the index i (S419), and the processing from step S407 is repeatedly performed.

[0138] By repeating these processes, the duplicate number calculation unit 160 stores the duplicate number of frames (concatenated images) in the horizontal direction of the furnace wall observation image I in the duplicate number storage array.

[0139] The multiple calculation process will be described using the simple example shown in FIG. 21. In the example shown in FIG. 21, the horizontal size of the concatenation image is 6 pixels. The multiple calculation unit 160 first adds "1" to the position of the multiple storage array corresponding to the horizontal pixel position of the concatenation image of the 0th frame (steps S411 to S417 in FIG. 20). Specifically, at the start of multiple calculation for the concatenation image of the 0th frame, since the translation amount variable j = 0 and the index k = 0, the starting position j1 for storing in the multiple storage array is the 0th position. Then, the multiple calculation unit 160 adds "1" to the values stored in the 0th to 5th positions of the multiple storage array and stores them while incrementing the index k one by one until k becomes 5.

[0140] After finishing the process for the concatenation image of the 0th frame, the multiple calculation unit 160 moves on to the process for the concatenation image of the next frame (executes the processes from step S419 in FIG. 20 to steps S407 to S417). At the start of multiple calculation for the concatenation image of the 1st frame, the translation amount variable j = 0 + the value stored in the 1st position of the determined movement amount storage array (i.e., the inter-frame movement amount of the 1st frame), and the index k = 0. In the example of FIG. 21, since "2" is stored in the 1st position of the determined movement amount storage array, for the concatenation image of the 1st frame, the starting position j1 for storing in the multiple storage array is the 2nd position. Then, the multiple calculation unit 160 adds "1" to the values stored in the 2nd to 7th positions of the multiple storage array and stores them while incrementing the index k one by one until k becomes 5.

[0141] After finishing the process for the concatenation image of the 1st frame, the multiple calculation unit 160 calculates the multiple in the same way for the concatenation image of the 2nd frame, the concatenation image of the 3rd frame, and so on. In this way, the multiple calculation unit 160 stores the multiple of the frames (concatenation images) in the horizontal direction of the furnace wall observation image in the multiple storage array.

[0142] [3-5. Furnace Wall Observation Image Generation Process] Returning to the description of FIG. 5, when the duplicate number calculation process (S40) is completed, the furnace wall observation image generation unit 150 performs a furnace wall observation image generation process (S50). Hereinafter, the furnace wall observation image generation process will be described based on FIGS. 22 to 30. FIG. 22 is a flowchart showing the overall flow of the furnace wall observation image generation process. FIG. 23 is a flowchart showing a process for securing a storage area for generating a furnace wall observation image. FIG. 24 is an explanatory diagram showing an outline of an image for calculating a representative value. FIG. 25 is a flowchart showing an example of a representative value calculation image generation process. FIG. 26 is an explanatory diagram for explaining the rearrangement of image data for calculating a representative value. FIG. 27 is an explanatory diagram showing a concatenation image constituting a furnace wall observation image. FIG. 28 is a flowchart showing an example of a storage process for an array for a furnace wall observation image. FIG. 29 is an explanatory diagram for explaining the storage process for an array for a furnace wall observation image. FIG. 30 is an image showing an example of a furnace wall observation image.

[0143] In the furnace wall observation image generation process, as described with reference to FIG. 2, a representative value of the pixel values of the overlapping portion Po is calculated from the concatenation image F including the portion (overlapping portion) Po that images the same position of the furnace wall 11, and the furnace wall observation image I is generated. Thereby, even when the fire powder is reflected in some of the concatenation images F, the influence can be reduced, and a clearer furnace wall observation image can be obtained.

[0144] In the furnace wall observation image generation process according to the present embodiment, as shown in FIG. 22, the furnace wall observation image generation unit 150 performs a process (S51) for securing a storage area for generating a furnace wall observation image, a representative value calculation image generation process (S53), and a storage process (S55) for an array for a furnace wall observation image. Hereinafter, each process will be described in detail.

[0145] (1) Process for Securing a Storage Area for Generating a Furnace Wall Observation Image First, the furnace wall observation image generation unit 150 executes the process for securing a storage area for generating a furnace wall observation image, as shown in FIG. 23.

[0146] The furnace wall observation image generation unit 150 sets initial values for the storage array for furnace wall observation images, the array for calculating representative values, the array for counting duplicates, and the variable j for the amount of translational movement (S510). The storage array for furnace wall observation images is an array that stores data related to the finally generated furnace wall observation images, and is prepared for the left region and the right region. The array for calculating representative values and the array for counting duplicates are arrays used in the representative value calculation image generation process described later. The array for calculating representative values and the array for counting duplicates are set to the same size as the horizontal size of the furnace wall observation image, and are prepared for the left region and the right region. For the storage array for furnace wall observation images, the array for calculating representative values, and the array for counting duplicates, a value such as "0" is stored as the initial value. The variable j for the amount of translational movement is a value (pixel value) indicating the start position of the concatenated image for each frame in the furnace wall observation image. The variable j for the amount of translational movement is also prepared for the left region and the right region.

[0147] Next, the furnace wall observation image generation unit 150 determines whether the variable j for the amount of translational movement is smaller than the size of the array for calculating representative values (S511). When the variable j for the amount of translational movement is smaller than the size of the array for calculating representative values (S511: YES), the furnace wall observation image generation unit 150 secures the j-th representative value calculation image and stores it in the j-th position of the array for calculating representative values (S512). The j-th representative value calculation image has a vertical size that is the same as the vertical size of the furnace wall observation image and a horizontal size that is the value (pixel value) stored in the j-th position of the duplicate count array.

[0148] For example, as shown in FIG. 24, in the 0th position of the representative value calculation array, an image is stored whose vertical size is the same as the vertical size of the furnace wall observation image and whose horizontal size is the size of the value (i.e., "1") stored in the 0th position of the multiple storage array. Also, for example, in the 2nd position of the representative value calculation array, an image is stored whose vertical size is the same as the vertical size of the furnace wall observation image and whose horizontal size is the size of the value (i.e., "2") stored in the 2nd position of the multiple storage array. That is, in the same position of the representative value calculation array, one or more images taken at the same position in the extrusion direction of the furnace wall are stored. Note that instead of storing the actual image, information for specifying the secured representative value calculation image may be stored in the representative value calculation array. Then, the furnace wall observation image generation unit 150 adds 1 to the parallel movement amount variable j (S513) and repeats the process from step S511.

[0149] The furnace wall observation image generation unit 150 repeatedly executes the processes of steps S511 and S513 while the parallel movement amount variable j is smaller than the size of the representative value calculation array. When it is determined in step S511 that the parallel movement amount variable j is equal to or greater than the size of the representative value calculation array (S511: NO), the furnace wall observation image generation unit 150 ends the process shown in FIG. 23.

[0150] (2) Representative value calculation image generation process After the furnace wall observation image generation unit 150 finishes the process for securing the storage area for generating the furnace wall observation image shown in FIG. 23, it executes the representative value calculation image generation process shown in FIG. 25.

[0151] The furnace wall observation image generation unit 150 first sets 0 to the index i for counting the number of frames read and sets 0 to the parallel movement amount variable j (S530), and determines whether the index i is less than the total number of frames (S531). If the index i is greater than or equal to the total number of frames (S531: NO), since the processes of steps S532 to S544 described later have been completed for all frames, the process shown in FIG. 25 ends.

[0152] On the other hand, when the index i is less than the total number of frames (S531: YES), the furnace wall observation image generation unit 150 reads the front view image Is (frame) of the furnace wall output from the inter-frame movement amount calculation unit 130 (S532), and adds the value of the i-th position of the determined movement amount storage array to the translation amount variable j (S533). Then, the furnace wall observation image generation unit 150 divides the frame into a left region and a right region using the symmetry axis C as in the inter-frame movement amount calculation process shown in FIG. 10 (S534), and performs perspective transformation on each of the left region and the right region of the frame to obtain a front view image of each region (S534). After that, the furnace wall observation image generation unit 150 cuts out the concatenation image F that constitutes the furnace wall observation image I from the obtained front view image Is of the furnace wall (S535). For example, as shown on the left side of FIG. 26, a concatenation image F of a predetermined horizontal size (fixed value) is cut out from the front view image Is of the furnace wall. The region of the concatenation image F is preset to a region where the furnace wall can be clearly confirmed in the front view image Is of the furnace wall. The position and size for cutting out the concatenation image F from the front view image Is of the furnace wall are the same for each frame.

[0153] After that, the furnace wall observation image generation unit 150 sets the index k representing the horizontal position of the concatenation image F to 0 (S537), and determines whether the index k is smaller than the horizontal size (pixel value) of the concatenation image (S538).

[0154] When the index k is smaller than the horizontal size of the concatenation image (S538: YES), the furnace wall observation image generation unit 150 adds the translation amount variable j and the index k to obtain the position j1 in the duplicate number storage array (S539). Then, the furnace wall observation image generation unit 150 sets the value stored in the j1-th position in the duplicate number storage array as j2 (S540), and stores the image data at the horizontal position k of the concatenation image F at the horizontal position j2 of the representative value calculation image of the j1-th position of the representative value calculation array secured by the process for securing the storage area for generating the furnace wall observation image shown in FIG. 23 (S541). After that, the furnace wall observation image generation unit 150 adds 1 to the j1-th position in the duplicate number storage array (S542), adds 1 to the index k (S543), and then repeats the process from step S538.

[0155] On the other hand, when it is determined in step S538 that the index k is equal to or greater than the horizontal size of the concatenation image (S538: NO), the furnace wall observation image generation unit 150 adds 1 to the index i and repeats the process from step S531.

[0156] By repeating these processes, the furnace wall observation image generation unit 150 stores the representative value calculation image cut out from the concatenation image of each frame in the representative value calculation array. That is, the representative value calculation image composed of the overlapping portions in each concatenation image is stored in the representative value calculation array. For example, in the example shown in FIG. 26, each concatenation image has a horizontal size of 6 pixels and is divided into 6 image data in pixel units in the horizontal direction. The furnace wall observation image generation unit 150 stores the image data of the concatenation image of the 0th frame in the 0th to 5th positions of the representative value calculation array secured by the process shown in FIG. 23. Further, the furnace wall observation image generation unit 150 stores the image data of the concatenation image of the 1st frame in the 2nd to 7th positions of the representative value calculation array. Similarly, the furnace wall observation image generation unit 150 stores the image data of the concatenation image of the 2nd frame, the concatenation image of the 3rd frame, ··· at predetermined positions in the representative value calculation array. In this way, the representative value calculation image is stored in the representative value calculation array.

[0157] (3) Storage process into the furnace wall observation image array When the furnace wall observation image generation unit 150 finishes the representative value calculation image generation process shown in FIG. 25, it executes the storage process into the furnace wall observation image array shown in FIG. 28.

[0158] The furnace wall observation image generation unit 150 first sets 0 to the horizontal position j of the furnace wall observation image (S550), and determines whether the horizontal position j of the furnace wall observation image is smaller than the size of the representative value calculation array (S551). If the horizontal position j of the furnace wall observation image is equal to or greater than the size of the representative value calculation array (S551: NO), after finishing the processes of steps S552 to S554 described later, since the furnace wall observation image I is generated, the process shown in FIG. 28 is terminated.

[0159] When the horizontal position j of the furnace wall observation image is smaller than the size of the array for representative value calculation (S551: YES), the index k representing the vertical position of the representative value calculation image is set to 0 (S552), and it is determined whether the index k is smaller than the vertical size (pixel value) of the representative value calculation image (S553). If the index k is greater than or equal to the vertical size of the representative value calculation image (S553: NO), the furnace wall observation image generation unit 150 adds 1 to the horizontal position j of the furnace wall observation image (S554), and then repeatedly executes the process from step S551.

[0160] On the other hand, when the index k is smaller than the vertical size of the representative value calculation image (S553: YES), the furnace wall observation image generation unit 150 calculates the representative value of the pixel value at the k-th vertical position for the representative value calculation image stored in the j-th position of the representative value calculation array, and stores the calculated representative value at the k-th vertical position and j-th horizontal position of the furnace wall observation image array (S555). For example, as shown in FIG. 29, it is assumed that a representative value calculation image composed of three overlapping image data is stored in the j-th position of the representative value calculation array. At this time, the furnace wall observation image generation unit 150 obtains the representative value of the three pixel values representing the same part of the furnace wall at all vertical positions of the representative value calculation image, and sets the representative value as the pixel value at the k-th vertical position and j-th horizontal position of the furnace wall observation image. After calculating the representative value of the pixel value at the k-th vertical position in step S555, the furnace wall observation image generation unit 150 adds 1 to the index k (S556), and then returns to the process of step S553.

[0161] The representative value may be, for example, the median or the average value of a plurality of pixel values representing the same part of the furnace wall. Further, the furnace wall observation image generation unit 150 may determine the value calculated as the representative value of the pixel value according to the number of overlaps of the concatenation images.

[0162] For example, in a portion where the number of duplicates is small, an average value, for example, may be used as a representative value. For example, immediately after the start of extrusion, the number of duplicates is small, but since the possibility of sparks is also low, it can be considered that using the average value as the representative value has no effect on the sparks. At a position where there is a possibility of sparks, for example, among a plurality of pixel values, those showing bright and high pixel values may be excluded and the average value may be calculated. Thereby, it is possible to suppress the image from becoming unclear due to the inclusion of sparks.

[0163] On the other hand, near the center of the extrusion position, the number of duplicates of the frames is around 10, and if sparks have occurred, it is about 1 or 2 frames out of these. From this, if the median value is used as the representative value, even if sparks are included in a part of the overlapping image data, it is possible to suppress the image from becoming unclear due to the inclusion of sparks.

[0164] In this way, by determining the value calculated as the representative value of the pixel values according to the number of duplicates of the concatenation images, more appropriate pixel values can be obtained, and a clear furnace wall observation image can be generated.

[0165] The furnace wall observation image generation unit 150 repeatedly performs the processes of steps S551 to S554 for the j-th representative value calculation image to generate image data at the horizontal position j of the furnace wall observation image. Then, image data at the horizontal position j of the furnace wall observation image is generated from all the representative value calculation images, and when the horizontal position j of the furnace wall observation image becomes equal to or greater than the size of the representative value calculation array in step S551, the furnace wall observation image I is generated.

[0166] As an example, FIG. 30 shows a furnace wall observation image (a. furnace wall observation image before processing) generated before applying the image processing method according to the present embodiment and a furnace wall observation image before processing (b. furnace wall observation image after processing) generated by applying the image processing method according to the present embodiment. As shown in FIG. 30, the influence of the powder of fire that appeared in the portion surrounded by the broken line of the furnace wall observation image before processing is reduced in the furnace wall observation image after processing. Thus, by using the image processing method according to the present embodiment, a clearer furnace wall observation image can be generated.

[0167] [4. Hardware Configuration] Based on FIG. 31, the hardware configuration of the image processing apparatus 100 according to the present embodiment will be described. FIG. 31 is a block diagram showing an example of the hardware configuration of an information processing apparatus 900 that functions as the image processing apparatus 100 according to the present embodiment.

[0168] The information processing apparatus 900 includes a processor (CPU 901 in FIG. 31), a ROM 903, and a RAM 905. The information processing apparatus 900 also includes a bus 907, an input I / F 909, an output I / F 911, a storage device 913, a drive 915, a connection port 917, and a communication device 919.

[0169] The CPU 901 functions as an arithmetic processing unit and a control unit. The CPU 901 controls all or part of the operations within the information processing apparatus 900 according to various programs recorded in the ROM 903, the RAM 905, the storage device 913, or the removable recording medium 925. The ROM 903 stores programs or arithmetic parameters used by the CPU 901. The RAM 905 temporarily stores programs used by the CPU 901 or parameters that change as appropriate during the execution of the programs. These are interconnected by a bus 907 constituted by an internal bus such as a CPU bus.

[0170] The bus 907 is connected to an external bus such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge.

[0171] The input I / F 909 is an interface that receives inputs from an input device 921, which is an operating means operated by a user, such as a mouse, keyboard, touch panel, button, switch, and lever. The input I / F 909 is configured as, for example, an input control circuit that generates an input signal based on information input by the user using the input device 921 and outputs it to the CPU 901. The input device 921 may be, for example, a remote control device using infrared rays or other radio waves, or an external device 927 such as a PDA corresponding to the operation of the information processing device 900. The user of the information processing device 900 can operate the input device 921 to input various data to the information processing device 900 or instruct processing operations.

[0172] The output I / F 911 is an interface that outputs the input information to an output device 923 that can notify the user visually or audibly. The output device 923 may be, for example, a display device such as a CRT display device, a liquid crystal display device, a plasma display device, an EL display device, and a lamp. Alternatively, the output device 923 may be an audio output device such as a speaker and headphones, a printer, a mobile communication terminal, a facsimile, or the like. The output I / F 911 instructs the output device 923 to output, for example, the processing results obtained by various processes executed by the information processing device 900. Specifically, the output I / F 911 instructs the display device to display the processing results by the information processing device 900 in text or image. Further, the output I / F 911 instructs the audio output device to convert an audio signal such as audio data received a playback instruction into an analog signal and output it.

[0173] The storage device 913 is one of the storage units of the information processing device 900 and is a device for storing data. The storage device 913 is composed of, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 913 stores programs executed by the CPU 901, various data generated by the execution of the programs, and various data acquired from the outside.

[0174] The drive 915 is a reader / writer for a recording medium and is built in or externally attached to the information processing device 900. The drive 915 reads the information recorded on the mounted removable recording medium 925 and outputs it to the RAM 905. Also, the drive 915 can write information to the mounted removable recording medium 925. The removable recording medium 925 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory. Specifically, the removable recording medium 925 may be a CD medium, a DVD medium, a Blu-ray (registered trademark) medium, a CompactFlash (registered trademark) (CF), a flash memory, an SD memory card (Secure Digital memory card), etc. Also, the removable recording medium 925 may be, for example, an IC card (Integrated Circuit card) or an electronic device equipped with a non-contact type IC chip.

[0175] The connection port 917 is a port for directly connecting a device to the information processing device 900. The connection port 917 is, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, an RS-232C port, etc. The information processing device 900 can directly acquire various data from the external device 927 connected to the connection port 917 or provide various data to the external device 927.

[0176] The communication device 919 is a communication interface composed of, for example, a communication device for connecting to a communication network 929. The communication device 919 is, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 919 may also be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication. The communication device 919 can transmit and receive signals, etc. in accordance with a predetermined protocol such as TCP / IP, for example, between the Internet and other communication devices. The communication network 929 connected to the communication device 919 is composed of a network connected by wire or wirelessly, etc. For example, the communication network 929 is the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication, etc.

[0177] As described above, an example of the hardware configuration of the information processing device 900 has been shown. Each of the above-described components may be configured using general-purpose members, or may be configured using hardware specialized for the functions of each component. The hardware configuration of the information processing device 900 can be appropriately changed according to the technical level at the time of implementing this embodiment.

[0178] As described above, the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present invention.

Explanation of Reference Numerals

[0179] 1 Coke oven 10 Carbonization chamber 11, 11L, 11R Furnace wall 13 Furnace bottom 15 Carbonization chamber opening 20 Extrusion ram beam 21 Extrusion ram 31, 33 Imaging device 100 Image processing device 110 Image acquisition unit 120 Preprocessing unit 130 Inter-frame movement amount calculation unit 140 Inter-frame movement amount estimation unit 150 Furnace wall observation image generation unit 160 Duplicate number calculation unit 300 Storage device 310 Captured image storage unit 320 Furnace wall observation image storage unit 500 Display device 900 Information processing device 907 Bus 913 Storage device 915 Drive 917 Connection port 919 Communication device 921 Input device 923 Output device 925 Removable recording medium 927 External device 929 Communication network A0 Partial area A1, A2 Template image C Symmetry axis Ia Temporal average image Ip Furnace wall perspective image (frame) IpR Right area of the furnace wall perspective image Is Furnace wall front view image Is1 First furnace wall front view image Is2 Second furnace wall front view image IsR Right area of the furnace wall front view image I Furnace wall observation image F Connecting image L Inter-frame movement amount R Template matching result S T Search area W Width of the temporal average image

Claims

1. An image processing apparatus for generating a furnace wall observation image for observing the inside of a furnace, for each of a plurality of furnace wall front view images obtained by converting at least a partial area of each of a plurality of perspective images sequentially captured by an imaging device moving inside the furnace into a front view image, a frame-to-frame movement amount calculation unit that calculates, as a frame-to-frame movement amount, a deviation amount between a first furnace wall front view image and a second furnace wall front view image, which are two of the furnace wall front view images including overlapping portions; a duplication number calculation unit that calculates, based on the frame-to-frame movement amount, the number of duplications of connection images cut out from the furnace wall front view image at each position in the horizontal direction of the furnace wall observation image; a furnace wall observation image generation unit that calculates a representative value of pixel values in an overlapping portion of the connection images based on the number of duplications of the connection images, and generates the furnace wall observation image using the representative value; a frame-to-frame movement amount estimation unit that calculates a frame-to-frame movement amount estimation value for correcting the frame-to-frame movement amount; and having, when the frame-to-frame movement amount calculated by template matching processing is outside a preset threshold range, the image processing apparatus that uses the frame-to-frame movement amount estimation value calculated by the frame-to-frame movement amount estimation unit as the frame-to-frame movement amount.

2. An image processing apparatus for generating a furnace wall observation image for observing the inside of a furnace, for each of a plurality of furnace wall front view images obtained by converting at least a partial area of each of a plurality of perspective images sequentially captured by an imaging device moving inside the furnace into a front view image, a frame-to-frame movement amount calculation unit that calculates, as a frame-to-frame movement amount, a deviation amount between a first furnace wall front view image and a second furnace wall front view image, which are two of the furnace wall front view images including overlapping portions; a duplication number calculation unit that calculates, based on the frame-to-frame movement amount, the number of duplications of connection images cut out from the furnace wall front view image at each position in the horizontal direction of the furnace wall observation image; a furnace wall observation image generation unit that calculates a representative value of pixel values in an overlapping portion of the connection images based on the number of duplications of the connection images, and generates the furnace wall observation image using the representative value; and having, the frame-to-frame movement amount calculation unit calculates the frame-to-frame movement amount based on an extrusion speed pattern of an extrusion ram in one coke extrusion, image processing apparatus.

3. The image processing apparatus according to claim 1, wherein the furnace wall observation image generation unit calculates a median value or an average value as a representative value of pixel values in an overlapping portion of the connection images.

4. The image processing apparatus according to claim 1, wherein the furnace wall observation image generation unit determines a value calculated as the representative value of the pixel values according to the number of overlaps of the connection images.

5. The image processing apparatus according to claim 2, wherein the furnace wall observation image generation unit calculates a median value or an average value as a representative value of pixel values in an overlapping portion of the connection images.

6. The image processing apparatus according to claim 2, wherein the furnace wall observation image generation unit determines a value calculated as the representative value of the pixel values according to the number of overlaps of the connection images.

7. An image processing method for generating a furnace wall observation image for observing the interior of a furnace, a frame - to - frame movement amount calculation step of calculating, as a frame - to - frame movement amount, a deviation amount between a first furnace wall front - view image and a second furnace wall front - view image, which are two furnace wall front - view images including overlapping portions with each other, among a plurality of furnace wall front - view images obtained by converting at least a partial region of each of a plurality of perspective images sequentially captured by an imaging device moving inside the furnace into a front - view image; a number - of - overlaps calculation step of calculating the number of overlaps of connection images cut out from the furnace wall front - view image at each position in the horizontal direction of the furnace wall observation image based on the frame - to - frame movement amount; a furnace wall observation image generation step of calculating a representative value of pixel values in an overlapping portion of the connection images based on the number of overlaps of the connection images and generating the furnace wall observation image using the representative value; a frame - to - frame movement amount estimation step of calculating an estimated value of the frame - to - frame movement amount for correcting the frame - to - frame movement amount; comprising The frame - to - frame movement amount calculation step uses, as the frame - to - frame movement amount, the estimated value of the frame - to - frame movement amount calculated by the frame - to - frame movement amount estimation step when the frame - to - frame movement amount calculated by template matching processing is outside a preset threshold range. The image processing method.

8. An image processing method for generating a furnace wall observation image for observing the interior of a furnace, For each of a plurality of front view images obtained by converting at least a partial region of each of a plurality of perspective images in which an imaging device moving inside the furnace sequentially images the furnace wall into a front view image, a frame-to-frame movement amount calculation step of calculating, as a frame-to-frame movement amount, a deviation amount between a first front view image of the furnace wall and a second front view image of the furnace wall, which are two of the front view images of the furnace wall and include overlapping portions with each other; A duplicate number calculation step of calculating, based on the frame-to-frame movement amount, the number of duplicates of connection images cut out from the front view image of the furnace wall at each position in the horizontal direction of the furnace wall observation image; A furnace wall observation image generation step of calculating a representative value of pixel values in the overlapping portion of the connection images based on the number of duplicates of the connection images, and generating the furnace wall observation image using the representative value; including; The frame-to-frame movement amount calculation step calculates the frame-to-frame movement amount based on an extrusion speed pattern of an extrusion ram in one coke extrusion, an image processing method.

9. A program for image processing of an image obtained by imaging the inside of a furnace, causing a computer to For each of a plurality of front view images obtained by converting at least a partial region of each of a plurality of perspective images in which an imaging device moving inside the furnace sequentially images the furnace wall into a front view image, a frame-to-frame movement amount calculation unit that calculates, as a frame-to-frame movement amount, a deviation amount between a first front view image of the furnace wall and a second front view image of the furnace wall, which are two of the front view images of the furnace wall and include overlapping portions with each other; A duplicate number calculation unit that calculates, based on the frame-to-frame movement amount, the number of duplicates of connection images cut out from the front view image of the furnace wall at each position in the horizontal direction of the furnace wall observation image; A furnace wall observation image generation unit that calculates a representative value of pixel values in the overlapping portion of the connection images based on the number of duplicates of the connection images, and generates the furnace wall observation image using the representative value; A frame-to-frame movement amount estimation unit that calculates an estimated value of the frame-to-frame movement amount for correcting the frame-to-frame movement amount; function as When the frame-to-frame movement amount calculated by template matching processing is outside a preset threshold range, the frame-to-frame movement amount calculation unit uses the estimated value of the frame-to-frame movement amount calculated by the frame-to-frame movement amount estimation unit as the frame-to-frame movement amount, a program.

10. A program for image processing of an image obtained by imaging the inside of a furnace, causing a computer to For each of a plurality of front view images of a furnace wall obtained by converting at least a partial region of each of a plurality of perspective images obtained by sequentially imaging the furnace wall with an imaging device moving inside the furnace, a frame-to-frame movement amount calculation unit that calculates a displacement amount between a first front view image of the furnace wall and a second front view image of the furnace wall, which are two of the front view images of the furnace wall and include overlapping portions, as a frame-to-frame movement amount; A duplication number calculation unit that calculates the number of duplications of connection images cut out from the front view image of the furnace wall at each position in the horizontal direction of the furnace wall observation image based on the frame-to-frame movement amount; A furnace wall observation image generation unit that calculates a representative value of pixel values in the overlapping portion of the connection images based on the number of duplications of the connection images and generates the furnace wall observation image using the representative value; Function as; The frame-to-frame movement amount calculation unit calculates the frame-to-frame movement amount based on the extrusion speed pattern of the extrusion ram in one coke extrusion. Program.

Images