Caving detection system and caving detection method

Abstract

This collapse detection system is provided with: an acquisition unit that acquires, at a first predetermined cycle, an image obtained by capturing an image of an object to be incinerated, which has been accumulated in a feeder of an incineration facility and is pushed into a combustion chamber; a first calculation unit that calculates a representative value based on the brightness of the first image acquired by the acquisition unit; a second calculation unit that calculates a representative value based on the brightness of two or more images captured within a time taken for one spallation of the object to be burned, from among a plurality of time-series images acquired by the acquisition unit prior to the first image; and a determination unit that performs a determination relating to the spallation on the basis of the representative value of the brightness calculated by the first calculation unit and the representative value of the brightness calculated by the second calculation unit.

Description

Technical Field

[0001] The present disclosure relates to a caving detection system and a caving detection method. This application claims the priority of Japanese Patent Application No. 2022-183989 filed on November 17, 2022, and incorporates its content herein by reference. Background Art

[0002] A supply amount detection system is disclosed in Patent Document 1, which includes: a photographing device configured to photograph an image of solid fuel before it is stacked on a feeding part of an incinerator and falls into a combustion chamber; and a detection device configured to detect the amount of the solid fuel supplied to the combustion chamber based on the image photographed by the photographing device. In this supply amount detection system, the amount of the solid fuel supplied to the combustion chamber is detected based on a difference value between a first luminance and a second luminance, where the first luminance is the luminance of the image at a first timing, and the second luminance is the luminance of the image at a second timing later than the first timing and lower than the first luminance. Specifically, the detection device divides the image into a plurality of divided images, counts the number of the divided images in which the difference value between the first luminance and the second luminance of each of the plurality of divided images exceeds a preset threshold value, and when the counted value exceeds a preset setting number, detects that the amount of the solid fuel supplied to the combustion chamber is excessive.

[0003] A fluidized bed incinerator is disclosed in Patent Document 2, which includes: a feeder; and a chute part connecting an outlet part of the garbage of the feeder to a supply port of the garbage of the fluidized bed incinerator. The fluidized bed incinerator is characterized in that a television camera is installed at a position where the fall of the garbage from the outlet part of the garbage of the feeder can be observed, and a calculation unit is provided for calculating the falling amount and the calorific value of the garbage based on the image of the installed television camera. In this fluidized bed incinerator, the amount of the garbage is detected by a processing flow including (a) taking in the image, (b) binarizing the image, (c) recognizing the contour, (d) calculating the area in the contour, (e) calculating the centroid in the area, (f) storing the centroid and the area, and (g) multiplying the moving distance between the centroid calculated last time and the centroid calculated this time by the area.

[0004] In Patent Document 3, an observation device for a combustion site that can visually observe a combustion site such as inside a boiler in operation is disclosed. In this observation device, there is a unit for performing grayscale conversion on an image obtained by photographing the combustion site, a unit for adjusting the contrast of an image obtained by photographing the combustion site, or a unit for performing contrast adjustment after performing grayscale conversion on an image obtained by photographing the combustion site. By performing predetermined processing on one image captured at a certain timing, information for visualizing the combustion site is generated.

[0005] Prior Art Documents

[0006] Patent Document 1: Japanese Patent No. 6979482 Gazette

[0007] Patent Document 2: Japanese Patent Application Laid-Open No. 9-060842

[0008] Patent Document 3: Japanese Patent Application Laid-Open No. 2019-196845 Summary of the Invention

[0009] Problems to be Solved by the Invention

[0010] However, due to complex phenomena occurring in the combustion chamber, it is sometimes difficult to appropriately detect the collapse of the object to be incinerated. However, in order to more appropriately perform control related to the combustion chamber, it is expected to improve the detection accuracy of the collapse of the object to be incinerated.

[0011] The present disclosure is completed to solve the above problems, and an object thereof is to provide a collapse detection system and a collapse detection method capable of improving the detection accuracy related to the collapse of the object to be incinerated.

[0012] Means for Solving the Problems

[0013] To solve the above problems, the collapse detection system of the present disclosure includes: an acquisition unit that acquires, at a first predetermined period, an image obtained by photographing an object to be incinerated that is stacked in a feeder of an incineration device and is pushed into a combustion chamber; a first calculation unit that calculates a representative value of the brightness based on a first image acquired by the acquisition unit; a second calculation unit that calculates a representative value of the brightness based on two or more images captured within the time taken for a single collapse of the object to be incinerated among a plurality of time-series images acquired by the acquisition unit before the first image; and a determination unit that makes a determination related to the collapse based on the representative value of the brightness calculated by the first calculation unit and the representative value of the brightness calculated by the second calculation unit.

[0014] The caving detection method of the present disclosure includes the following steps: one or more computers acquire images obtained by photographing the objects to be incinerated stacked in the feeder of the incineration device and pushed into the combustion chamber at a first predetermined period; calculate a representative value based on the brightness of the acquired first image; calculate representative values based on the brightness of two or more images captured within the time taken for one caving of the objects to be incinerated among a plurality of images in the time series acquired before the first image; and make a determination related to the caving based on the representative value based on the brightness of the first image and the representative values based on the brightness of the two or more images.

[0015] The caving detection system of the present disclosure includes: an acquisition unit that acquires images obtained by photographing the objects to be incinerated stacked in the feeder of the incineration device and pushed into the combustion chamber at a first predetermined period; and a caving detection unit that makes a determination related to the caving based on the first image acquired by the acquisition unit, i.e., the first input element, and two or more images captured within the time taken for one caving of the objects to be incinerated among a plurality of images in the time series acquired by the acquisition unit before the first image, i.e., the second input element.

[0016] Advantageous Effects of the Invention

[0017] According to the present disclosure, it is possible to provide a caving detection system and a caving detection method capable of improving the detection accuracy related to the caving of the objects to be incinerated. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic structural diagram showing the whole incineration device according to an embodiment of the present disclosure.

[0019] Figure 2 It is a functional block diagram of the information processing device according to an embodiment of the present disclosure.

[0020] Figure 3 It is a diagram showing an example of an image to be determined by the first determination unit according to the first embodiment of the present disclosure.

[0021] Figure 4 It is a diagram showing an example of an image to be determined by the third determination unit according to the first embodiment of the present disclosure.

[0022] Figure 5 It is a diagram showing an example of the determination result in the time series of the first determination unit according to the first embodiment of the present disclosure.

[0023] Figure 6 It is a diagram illustratively showing a list of images used for learning by the second determination learning-complete model according to the first embodiment of the present disclosure.

[0024] Figure 7This is a diagram showing an example of the determination result over time of the second determination unit according to the first embodiment of the present disclosure.

[0025] Figure 8 This is a diagram showing an example of an image that is the object of determination by the interference determination unit according to the first embodiment of the present disclosure.

[0026] Figure 9 This is a diagram showing an example of a low-deviation image and a high-deviation image used for learning by the learned model for interference determination according to the first embodiment of the present disclosure.

[0027] Figure 10 This is a diagram showing at a glance the images used for learning by the learned model for interference determination according to the first embodiment of the present disclosure in a form corresponding to the magnitude of the deviation from the brightness.

[0028] Figure 11 This is a diagram showing an example of the determination result over time of the interference determination unit according to the first embodiment of the present disclosure.

[0029] Figure 12 This is a diagram showing an example of the determination result over time of the third determination unit according to the first embodiment of the present disclosure.

[0030] Figure 13 This is a flowchart showing an example of the operation of the information processing apparatus according to the first embodiment of the present disclosure.

[0031] Figure 14 This is a diagram showing an example of the determination result over time of the first collapse determination unit and the second collapse determination unit according to the first embodiment of the present disclosure.

[0032] Figure 15 This is a diagram for explaining the binarized data according to the second embodiment of the present disclosure.

[0033] Figure 16 This is a diagram for conceptually explaining the method of calculating the similarity of the second collapse determination unit according to the second embodiment of the present disclosure.

[0034] Figure 17 This is a flowchart showing an example of the operation of the information processing apparatus according to the second embodiment of the present disclosure.

[0035] Figure 18 This is a diagram showing an example of the determination result over time of the first collapse determination unit and the second collapse determination unit according to the second embodiment of the present disclosure.

[0036] Figure 19 This is a functional block diagram of the information processing apparatus according to the third embodiment of the present disclosure.

[0037] Figure 20This is a diagram showing an example of an image that is the object of calculation by the first calculation unit according to the third embodiment of the present disclosure.

[0038] Figure 21 This is a diagram illustratively showing a list of images used for learning by the fourth determination learned model according to the third embodiment of the present disclosure.

[0039] Figure 22 This is a table showing the numerical range of the total amount of brightness change that is the determination reference by the third collapse determination unit and the determination results of the fourth and fifth determination units according to the third embodiment of the present disclosure.

[0040] Figure 23 This is a flowchart showing an example of the operation of the information processing device according to the third embodiment of the present disclosure.

[0041] Figure 24 This is a hardware structure diagram showing the structure of a computer according to an embodiment of the present disclosure. Detailed Embodiment

[0042] Hereinafter, a method for implementing an incineration device and a collapse detection system will be described with reference to the accompanying drawings.

[0043] <First Embodiment of Incineration Device>

[0044] The incineration device 100 is, for example, a grate-type waste incinerator that incinerates municipal waste, industrial waste, biomass, etc. as the object to be incinerated. Hereinafter, the object to be incinerated may sometimes be referred to as "waste". That is, in the present embodiment, the waste is a fuel for generating a combustion reaction in the incineration device. As Figure 1 shown, the incineration device 100 includes, for example: a hopper 102, a feeder 104, a furnace main body 108, a pusher device 110, an air supply device 112, a heat recovery boiler 114, a cooling tower 116, a dust collection device 118, a chimney 120, and a collapse detection system 1.

[0045] The feeder 104 is a passage extending toward the combustion chamber R of the furnace main body 108. The waste Fg introduced from the hopper 102 is guided into the feeder 104 and temporarily accumulated. The furnace main body 108 has a combustion chamber R inside for incinerating the waste Fg. When the direction of conveying the waste Fg in the furnace main body 108 is set as the conveying direction W1 ( Figure 1 the left-right direction in the figure), the downstream end 121 on the downstream side in the conveying direction W1 of the feeder 104 is connected to the receiving port 122 of the combustion chamber R.

[0046] The ejecting device 110 has an ejecting arm 124 for ejecting the refuse Fg accumulated in the feeder 104 toward the combustion chamber R through the receiving port 122. The ejecting arm 124 can move within the feeder 104 from the upstream side to the downstream side and from the downstream side to the upstream side in the conveying direction W1 (it can move forward and backward). In the present embodiment, the ejecting arm 124 reciprocates within the feeder 104 along the conveying direction W1 to intermittently supply the refuse Fg into the combustion chamber R. In the present embodiment, the ejecting arm 124 is an example of the controlled device S.

[0047] The furnace main body 108 includes grate bars 126 (grates), and the refuse Fg ejected into the combustion chamber R through the receiving port 122 falls onto the grate bars 126 (grates). The grate bars 126 correspond to the bed portion in the combustion chamber R. The grate bars 126 move the refuse Fg on the grate bars 126 in a direction away from the receiving port 122 (the downstream side in the conveying direction W1). The grate bars 126 are an example of the controlled device S. In addition, the combustion chamber R includes a drying region 128, a combustion region 130, and a post-combustion region 132 arranged in sequence from the upstream side to the downstream side in the conveying direction W1. The drying region 128 dries the refuse Fg by the heat in the combustion chamber R. The combustion region 130 ignites a flame 131 to burn the refuse Fg. The post-combustion region 132 completely burns the combustion residues that are not completely burned in the combustion region 130. The refuse Fg after drying, burning, and post-combusting in the combustion chamber R becomes ash 135, which falls from an ash chute 146 located downstream of the post-combustion region 132 and is discharged to the outside of the furnace main body 108.

[0048] The air supply device 112 supplies primary air for the combustion of the refuse Fg and secondary air for reducing the concentration of unburned gases such as carbon monoxide generated by the combustion of the refuse Fg to the combustion chamber R. The air supply device 112 has: an air supply pipe 136, a blower 138 provided on the air supply pipe 136, and a first flow rate adjustment valve 140 and a second flow rate adjustment valve 142 provided on the air supply pipe 136. A part of the air pressurized by the blower 138 and flowing in the air supply pipe 136 is adjusted in flow rate by the first flow rate adjustment valve 140 as primary air and is supplied into the combustion chamber R through the grate bars 126 from the lower part of the combustion chamber R. The remaining part of the air flowing in the air supply pipe 136 is adjusted in flow rate by the second flow rate adjustment valve 142 as secondary air and is supplied from the side wall of the combustion chamber R to the upper side within the combustion chamber R. In the present embodiment, for example, primary air is supplied to the drying region 128, the combustion region 130, and the post-combustion region 132 of the combustion chamber R respectively, and secondary air is supplied to the upper side of the combustion region 130. In the present embodiment, the blower 138, the first flow rate adjustment valve 140, and the second flow rate adjustment valve 142 are all examples of the controlled device S.

[0049] The heat recovery boiler 114, the cooling tower 116, the dust collection device 118, and the chimney 120 are respectively provided in the flue 144 through which the exhaust gas 143 generated by burning the waste Fg in the combustion chamber R flows. The exhaust gas 143 flows in the order of the heat recovery boiler 114, the cooling tower 116, the dust collection device 118, and the chimney 120. The heat recovery boiler 114 generates steam by the thermal energy of the exhaust gas 143. The cooling tower 116 reduces the temperature of the exhaust gas 143 that has passed through the heat recovery boiler 114. The dust collection device 118 captures the fly ash contained in the exhaust gas 143 that has passed through the cooling tower 116. The chimney 120 discharges the exhaust gas 143 that has passed through the dust collection device 118 to the outside of the incinerator 100. The steam generated in the heat recovery boiler 114 is supplied to, for example, a steam turbine (not shown) disposed outside the incinerator 100.

[0050] <Collapse detection system>

[0051] The collapse detection system 1 detects the situation where the waste Fg accumulated in the feeder 104 is pushed into the combustion chamber R by the pushing arm 124 and supplied to the combustion chamber R. Specifically, the collapse detection system 1 detects the collapse of the waste Fg from the feeder 104 into the combustion chamber R. Here, the "collapse" means, for example, supplying a certain amount of the waste Fg that has accumulated in the feeder 104 to the combustion chamber R at one time. In the present embodiment, the collapse is classified into a collapse of the first scale and a collapse of the second scale having a collapse scale larger than that of the collapse of the first scale.

[0052] The collapse of the first scale includes the following situations: It is impossible to visually confirm the layer structure of about one-third of the waste Fg in the width direction (the direction orthogonal to the conveying direction W1) of the furnace main body 108 in the entire waste Fg accumulated in the feeder 104 in the infrared image captured by the imaging device 2 described later, and a part of the waste Fg after the collapse scatters in the combustion chamber R. In addition, the collapse of the first scale also includes the following situations: It is impossible to visually confirm the layer structure of more than one-third and less than two-thirds of the waste Fg in the width direction of the furnace main body 108 in the entire waste Fg accumulated in the feeder 104 in the infrared image, and a part of the waste Fg after the collapse does not scatter in the combustion chamber R.

[0053] On the other hand, the collapse of the second scale includes the following situations: It is impossible to visually confirm the layer structure of more than two-thirds of the waste Fg in the width direction of the furnace main body 108 in the entire waste Fg accumulated in the feeder 104 in the infrared image, and a part of the waste Fg after the collapse scatters in the combustion chamber R. In addition, the collapse of the second scale also includes the following situations: The waste Fg falls onto the flame 131 in the visible light image captured by the imaging device 2 described later, and the flame 131 disappears in an area of more than one-third in the width direction of the furnace main body 108.

[0054] Therefore, in the present embodiment, the caving detection system 1 detects the scale (quantity) of the refuse Fg caving into the combustion chamber R. As Figure 1 and Figure 2 shown, the caving detection system 1 includes, for example, an imaging device 2 and an information processing device 4.

[0055] [Imaging device]

[0056] The imaging device 2 images the inside of the combustion chamber R in such a way that the refuse Fg accumulated in the feeder 104 is imaged. The image of the refuse Fg captured by the imaging device 2 is sent to the information processing device 4 in real time. The imaging device 2 is disposed in the furnace main body 108 to image the front surface Fr of the refuse Fg facing the combustion chamber R before the refuse Fg caves into the combustion chamber R. Specifically, the imaging device 2 is provided at the furnace tail 145 of the furnace main body 108, and the furnace tail 145 of the furnace main body 108 is located downstream of the post-combustion region 132 in the combustion chamber R in the conveying direction W1. In addition, as long as the infrared image and the visible light image of the front surface Fr of the refuse Fg can be captured, the imaging device 2 may be provided at a part of the furnace main body 108 other than the furnace tail 145.

[0057] In the present embodiment, the imaging device 2 has an infrared camera 5 capable of capturing an infrared image and a visible light camera 6 capable of capturing a visible light image (see Figure 1 ). The imaging device 2 can capture the infrared image and the visible light image of the front surface Fr of the refuse Fg extending downstream in the conveying direction W1 from the receiving port 122 of the combustion chamber R. The infrared camera 5 captures the front surface Fr of the refuse Fg in a wavelength band of, for example, 3.8 μm to 4.2 μm to generate an infrared image. Since the infrared camera 5 captures images in the above wavelength band, it can penetrate the flame 131, and the flame 131 can be prevented from being imaged in the generated infrared image. The visible light camera 6 captures the front surface Fr of the refuse Fg in a predetermined wavelength band in the visible wavelength region to generate a visible light image. Since the visible light camera 6 captures images in the above wavelength band, it cannot penetrate the flame 131, and the flame 131 is mainly imaged in the generated visible light image.

[0058] [Information processing device]

[0059] The information processing device 4 detects information related to the refuse Fg supplied from the inside of the feeder 104 to the combustion chamber R based on the image captured by the imaging device 2. As Figure 2 shown, the information processing device 4 includes, for example: an acquisition unit 40, a caving detection unit 41, a control unit 80, and a storage unit 90.

[0060] (Structure of the acquisition unit)

[0061] The acquisition unit 40 acquires the above-mentioned images over time by receiving in real time the images transmitted from the imaging device 2. In the present embodiment, the acquisition unit 40 acquires infrared images from the imaging device 2 at a first predetermined period and acquires visible light images from the imaging device 2 at a second predetermined period. The first predetermined period and the second predetermined period are determined based on, for example, the frame rate (fps: frames per second) of the imaging device 2. The acquisition unit 40 transmits the acquired infrared images and visible light images to the collapse detection unit 41.

[0062] (Structure of the collapse detection unit)

[0063] Based on the infrared images and visible light images received from the acquisition unit 40, the collapse detection unit 41 detects the situation where the garbage Fg collapses from the feeder 104 into the combustion chamber R and the scale (quantity) of the garbage Fg that has collapsed and is supplied to the combustion chamber R. The collapse detection unit 41 has, for example: a first calculation unit 50, a second calculation unit 55, a third calculation unit 60, a fourth calculation unit 65, and a determination unit 70.

[0064] (First calculation unit)

[0065] The first calculation unit 50 receives the infrared image among the images received from the acquisition unit 40, and calculates a first feature amount based on the received infrared image. In the present embodiment, the first feature amount is a representative value of the brightness based on one infrared image (for example, the most recent infrared image) received by the first calculation unit 50. Hereinafter, the infrared image used by the first calculation unit 50 to calculate the representative value of the brightness is referred to as the "first infrared image". That is, the first calculation unit 50 calculates the representative value of the brightness based on the first infrared image acquired by the acquisition unit 40. The first infrared image is an example of the first image. In the present embodiment, the first calculation unit 50 calculates the representative value of the brightness in a specific object area that is a part of the first infrared image. Hereinafter, this object area is referred to as the "first object area 42".

[0066] As Figure 3 shown, the first calculation unit 50 calculates the representative value of the brightness with the area in the first infrared image that mainly reflects the garbage Fg piled up in the feeder 104 as the first object area 42. In the present embodiment, the first object area 42 that is the object of calculation by the first calculation unit 50 is divided into a plurality of areas. In Figure 3 it, as an example, shows the case where the first object area 42 is equally divided into 9 areas in a 3×3 matrix shape. In addition, the first object area 42 is not limited to the case of being equally divided into 9 areas in a matrix shape, and may be divided into 2 to 8 areas or 10 or more areas. Hereinafter, Figure 3The first object region 42 shown in [figure] is successively referred to as "first region 42a", "second region 42b", "third region 42c", "fourth region 42d", "fifth region 42e", "sixth region 42f", "seventh region 42g", "eighth region 42h", and "ninth region 42i" from the upper left to the lower right. The first calculation unit 50, for example, calculates the average value of the brightness of each of the first region 42a to the ninth region 42i of the first object region 42 in the first infrared image, and calculates the average value of the entire first object region 42 obtained by further averaging the calculated average values of the nine brightness values as the representative value of the brightness based on the first infrared image. In addition, the representative value calculated by the first calculation unit 50 is not limited to the average value, and may be a statistic such as the median, for example. That is, the calculation method of the representative value of the first calculation unit 50 is not necessarily limited to the above. The first calculation unit 50 sends the calculated representative value of the brightness of the first object region 42 to the determination unit 70.

[0067] (Second calculation unit)

[0068] The second calculation unit 55 receives the infrared image in the image received from the acquisition unit 40, and calculates the second feature amount based on the received plurality of infrared images. In the present embodiment, the second feature amount is the representative value of the brightness based on the plurality of infrared images received by the second calculation unit 55. Specifically, the second calculation unit 55 calculates the representative value of the brightness based on a plurality of infrared images taken within the time taken for one collapse of the garbage Fg in two or more infrared images in the time series obtained before the first infrared image used by the first calculation unit 50 to calculate the representative value. The "time taken for one collapse" mentioned here, for example, is the time obtained by averaging the plurality of obtained times after obtaining the time when it is judged by a person that one collapse of the garbage Fg has occurred multiple times. Specifically, the average number of frames is calculated based on the distribution of the number of frames (number of images) of the imaging device 2 indicating one collapse, etc., thereby defining the time taken for one collapse of the garbage Fg.

[0069] The two or more infrared images used by the second calculation unit 55 to calculate the representative value of the brightness are infrared images taken at a predetermined time interval. Hereinafter, the time interval between two or more infrared images is referred to as "first time". In the present embodiment, the second calculation unit 55 calculates the representative value of the brightness based on each infrared image in the infrared images obtained by the acquisition unit 40 per unit time. Therefore, the first time is the above-mentioned first predetermined period. The first time is, for example, a length less than 1 second.

[0070] In the present embodiment, two or more infrared images are infrared images captured before the first infrared image in a second time that is at least twice the first time. The second calculation unit 55 calculates a representative value of the brightness based on multiple infrared images, for example, based on three or more (more specifically, four or more) infrared images out of the two or more infrared images. Here, assuming that the time taken for a single collapse of the garbage Fg is S, the first predetermined period is T i , and the number of infrared images (number of frames) used for the calculation is N i , the following formula (i) holds

[0071] S / 2 < T i ×N i …(i)

[0072] That is, the multiple infrared images are multiple images captured within a time that is at least longer than half of S. In the present embodiment, multiple infrared images are captured within the time of S. Hereinafter, the multiple infrared images used by the second calculation unit 55 to calculate the representative value of the brightness are each referred to as a "second infrared image". The second infrared image is an example of the second image

[0073] The second calculation unit 55 calculates a representative value of the brightness of a specific object area that is part of the second infrared images. The object area that is the object of calculation by the second calculation unit 55 is the same area as the above-described first object area 42. The second calculation unit 55 calculates the average value of the brightness of each of the first area 42a to the ninth area 42i of the first object area 42, and calculates the average value of the entire first object area 42 obtained by further averaging the calculated average values of the 9 brightness values. In addition, the second calculation unit 55 calculates the average value of the entire multiple first object areas 42 based on the multiple second infrared images as the representative value. Additionally, the representative value calculated by the second calculation unit 55 is not limited to the average value, and may be a statistic such as the median, for example. That is, the calculation method of the representative value of the second calculation unit 55 is not necessarily limited to the above. The second calculation unit 55 sends the representative value of the brightness of the entire multiple first object areas 42 based on the multiple second infrared images to the determination unit 70

[0074] (Third calculation unit)

[0075] The third calculation unit 60 receives the visible light image in the image received from the acquisition unit 40, and calculates a third feature amount based on the received visible light image. In the present embodiment, the third feature amount is a representative value of the luminance of one visible light image (for example, the latest visible light image) received by the third calculation unit 60. Hereinafter, the visible light image used by the third calculation unit 60 to calculate the representative value of the luminance is referred to as the "first visible light image". That is, the third calculation unit 60 calculates the representative value of the luminance based on the first visible light image acquired by the acquisition unit 40. The first visible light image is another example of the first image. Figure 4 An example of the visible light image received by the third calculation unit 60 is shown. In Figure 4 order to facilitate illustration, the first visible light image (a) is represented in monotone (black and white display). As Figure 4 shown, in the present embodiment, the third calculation unit 60 generates an image (b) in which only the red component in the monochromatic components is extracted from the first visible light image (a), and calculates the representative value of the luminance in a specific object region, which is a part of the image (b) in which only the red component is extracted. Hereinafter, the image in which only the red component is extracted from the first visible light image by the third calculation unit 60 is referred to as the "monochromatic component image". In addition, the object region is referred to as the "second object region 43". The monochromatic component image is an example of the visible light image. In addition, the monochromatic component extracted from the first visible light image by the third calculation unit 60 is not limited to the red component. For example, only the green component or only the blue component may be used as the monochromatic component, and the monochromatic component image is generated according to the first visible light image.

[0076] The third calculation unit 60 calculates the representative value of the luminance by using the region in the monochromatic component image where the flame 131 is mainly reflected as the second object region 43. Here, the second object region 43 that is the object of calculation by the third calculation unit 60 is equally divided into a plurality of regions. In Figure 4 order to facilitate illustration, the second object region 43 is divided into three regions in the width direction (the direction orthogonal to the conveying direction W1) of the furnace main body 108. In addition, the second object region 43 is not limited to the case of being equally divided into three regions, and may be divided into two regions or four or more regions. Hereinafter, Figure 4The second object area 43 shown in the figure is successively called the "left area 43l", the "central area 43c", and the "right area 43r" from left to right. The third calculation unit 60 calculates the median value of the brightness of each of the left area 43l, the central area 43c, and the right area 43r in the second object area 43 in the monochromatic component image as a representative value. In addition, the representative value calculated by the third calculation unit 60 is not limited to the median value, and may be a statistic such as an average value, for example. That is, the calculation method of the representative value of the third calculation unit 60 is not necessarily limited to the above. The third calculation unit 60 sends the representative values of the brightness of each of the left area 43l, the central area 43c, and the right area 43r in the calculated monochromatic component image to the determination unit 70.

[0077] (Fourth calculation unit)

[0078] The fourth calculation unit 65 receives the visible light image in the image received from the acquisition unit 40, and calculates the fourth feature amount based on the plurality of received visible light images. In the present embodiment, the fourth feature amount is the representative value of the brightness of the plurality of visible light images received by the fourth calculation unit 65. Specifically, the fourth calculation unit 65 calculates the representative value of the brightness of the plurality of visible light images taken within the time taken for one collapse of the garbage Fg in two or more visible light images in the time series acquired before the first visible light image used by the third calculation unit 60 for calculating the representative value.

[0079] The two or more visible light images used by the fourth calculation unit 65 in the calculation of the representative value of the brightness are visible light images taken at a predetermined time interval from each other. Hereinafter, the time interval between two or more visible light images is called the "third time". In the present embodiment, the fourth calculation unit 65 calculates the representative value of the brightness based on each of the visible light images obtained by the acquisition unit 40 per unit time. Therefore, the third time is the above-mentioned second predetermined period. The third time is, for example, a length less than 1 second.

[0080] In the present embodiment, the two or more visible light images are images taken before the first visible light image in a fourth time that is at least twice the third time. As the two or more visible light images, the fourth calculation unit 65 calculates the representative value of the brightness based on, for example, three or more (more specifically, four or more) visible light images. In the present embodiment, the fourth calculation unit 65 calculates the representative value of the brightness based on each of the images obtained by the acquisition unit 40 per unit time. Here, when the time taken for one collapse of the garbage Fg is set to S, the second predetermined period is set to T ii , the number of visible light images (number of frames) used for the calculation is set to N ii , the following formula (ii) holds.

[0081] S / 2 < T ii ×N ii …(ii)

[0082] That is, the multiple visible light images are multiple images obtained in a time period that is at least longer than half of the time S. In the present embodiment, the multiple visible light images are obtained within the time period of S. Hereinafter, the multiple images used by the fourth calculation unit 65 to calculate the representative value of the luminance are each referred to as a "second visible light image". The second visible light image is an example of the second image.

[0083] The fourth calculation unit 65 generates a monochromatic component image based on the second infrared image, and calculates the representative value of the luminance of a specific object area that is a part of the monochromatic component image. The object area that is the object of calculation by the fourth calculation unit 65 is the same area as the second object area 43 described above. The fourth calculation unit 65 calculates the median value of the luminance of each of the left area 43l, the central area 43c, and the right area 43r of the second object area 43. In addition, the fourth calculation unit 65 calculates the median value of the entire left area 43l, the entire central area 43c, and the entire right area 43r as the representative value based on the multiple monochromatic component images. In addition, the calculation method of the representative value calculated by the fourth calculation unit 65 is not limited to the median value, and may be a statistic such as an average value, for example. That is, the calculation method of the representative value of the fourth calculation unit 65 is not necessarily limited to the above. The fourth calculation unit 65 sends the representative values of the luminance of the entire multiple left areas 43l, the entire central areas 43c, and the entire right areas 43r based on the calculated multiple monochromatic component images to the determination unit 70.

[0084] (Determination unit)

[0085] Return Figure 2 Next, the structure of the determination unit 70 will be described. The determination unit 70 performs various determination processes (described later) based on the images received from the acquisition unit 40 and the feature amounts (for example, the representative values of the luminance) received from the first calculation unit 50, the second calculation unit 55, the third calculation unit 60, and the fourth calculation unit 65, respectively. In the present embodiment, the determination unit 70 has, for example: a first determination unit 71, a second determination unit 72, an interference determination unit 74, a third determination unit 73, a first collapse determination unit 75, and a second collapse determination unit 76.

[0086] (First determination unit)

[0087] The first determination unit 71 makes a determination related to caving based on the representative value of luminance (first feature quantity) received from the first calculation unit 50 and the representative value of luminance (second feature quantity) received from the second calculation unit 55. Hereinafter, the determination made by the first determination unit 71 is referred to as "first determination". The first determination unit 71 calculates the feature change amount V1 (absolute value) generated between the first infrared image that is the object of the first determination and a plurality of second infrared images according to the following formula (iii).

[0088] V1 = |A m (t) - (ΣB m (t s )) / s|...(iii)

[0089] A m (t) in the above formula (iii) is the representative value of luminance (first feature quantity) calculated by the first calculation unit 50 at time t. ΣB m (t s ) is the representative value of luminance (second feature quantity) calculated by the second calculation unit 55. t s is a plurality of times at which each second infrared image is acquired, and time t s corresponds one by one to one second infrared image. s is the number of second infrared images and is the number of frames acquired per unit time.

[0090] The first determination unit 71 makes a determination as the first determination whether the calculated feature change amount V1 is equal to or greater than a predetermined first threshold value. When the calculated feature change amount V1 is equal to or greater than the first threshold value, the first determination unit 71 determines that there is a possibility of caving. When the feature change amount V1 is less than the first threshold value, the first determination unit 71 determines that there is no possibility of caving. In the present embodiment, when the feature change amount V1 is equal to or greater than the first threshold value, the first determination unit 71 outputs a flag indicating that there is caving (i = 1). On the other hand, when the feature change amount V1 is less than the first threshold value, the first determination unit 71 outputs a flag indicating that there is no caving (i = 0). Hereinafter, the flag (i = 1 or 0) related to the determination of caving output by the first determination unit 71 is referred to as "first caving determination flag". That is, the first caving determination flag indicates the result of the first determination and represents a value of 1 or 0. In addition, the first threshold value is stored in the storage unit 90 in advance. The first determination unit 71 makes the first determination by appropriately referring to the first threshold value stored in the storage unit 90. The first determination unit 71 sends the first caving determination flag, which is the result of the first determination, to the first caving determination unit 75 and the second caving determination unit 76.

[0091] Here, Figure 5 shows the change in the time series of the feature change amount V1 in a specific time period and the change in the time series of the first caving determination flag corresponding to the feature change amount V1 in the same time period.Figure 5 The graph shown in Figure 5 (b) of Figure 5 shows the moment when the operator (skilled operator) of the incinerator 100 visually inspects the inside of the combustion chamber R and determines that there is a collapse of the garbage Fg. In addition, the time corresponding to the intervals between the vertically extending time scale lines (dashed lines) in the graph shown in Figure 5 is equal in any interval. Based on the results shown in

[0092] (Second determination unit)

[0093] The second determination unit 72 makes a determination related to the collapse based on one or more infrared images received from the acquisition unit 40. In the present embodiment, the second determination unit 72 uses one first infrared image (the latest infrared image) that is the object of calculation by the first calculation unit 50 among the images received from the acquisition unit 40 to make a determination related to the collapse. In addition, when making a determination of the collapse, the second determination unit 72 may also use an image different from the first infrared image among the images received from the acquisition unit 40. In addition, when making a determination of the collapse, the second determination unit 72 may use a plurality of images, or may include the first infrared image among the plurality of images.

[0094] The second determination unit 72 makes a determination using a learned model that has been learned to output a determination result related to the possibility of a collapse when the first infrared image is input. Hereinafter, the determination by the second determination unit 72 is referred to as "second determination", and the learned model used by the second determination unit 72 in the second determination is referred to as "second determination learned model 91". The second determination learned model 91 is pre-stored in the storage unit 90 (refer to Figure 2). The second determination unit 72 obtains the output determination result as the result of the second determination by inputting the first infrared image received from the acquisition unit 40 into the second determination learned model 91 stored in the storage unit 90. The second determination learned model 91 is, for example, a deep learning model (supervised learning model) such as a convolutional neural network (CNN: Convolutional Neural Network). The second determination learned model 91 is generated (learned) by repeatedly performing a learning step multiple times (for example, thousands of times), where the infrared image captured by the imaging device 2 is input, and the presence or absence of collapse corresponding to the infrared image (correct solution data judged by a person) is taught. In addition, the second determination learned model 91 may use a recurrent neural network (RNN: Recurrent Neural Network) instead of the CNN.

[0095] Here, image examples (a, d) showing that the garbage Fg flies (collapse occurs) in the infrared image captured by the imaging device 2, image example 1 (b, e) showing no collapse (no collapse), and image example 2 (c, f) are classified into cases where it is easy to see and cases where it is not easy to see inside the combustion chamber R and shown in Figure 6 . As Figure 6 indicated by the double-dashed lines in each image example in, in the present embodiment, the second determination unit 72 inputs the infrared image of the target region, which is a part of the first infrared image, into the second determination learned model 91. Hereinafter, this target region is referred to as the "third target region 44". The third target region 44 is, for example, a region that is mostly in the upper half of the first infrared image and is a region that captures all or most of the garbage Fg stacked in the feeder 104. In addition, as Figure 6 shown in the image example (a) in, when the garbage Fg has collapsed, a part (or parts) of the collapsed garbage Fg temporarily scatters (scatters) inside the combustion chamber R, and sometimes the layer structure of the garbage Fg stacked in the feeder 104 cannot be visually confirmed. By Figure 6 inputting the infrared images of the plurality of third target regions 44 shown into the second determination learned model 91 and teaching the presence or absence of collapse (correct solution data) corresponding to each infrared image, the second determination learned model 91 is pre-generated. When the learned second determination learned model 91 is input with a new infrared image, it outputs a value related to the possibility of collapse corresponding to the infrared image. Hereinafter, the value output by the second determination learned model 91 is referred to as the "determination score".

[0096] The second determination unit 72 makes a second determination as to whether the determination score obtained from the second learned model 91 for determination is equal to or greater than a predetermined second threshold value. When the obtained determination score is equal to or greater than the second threshold value, the second determination unit 72 determines that there is a possibility of collapse. When the determination score is less than the second threshold value, the second determination unit 72 determines that there is no possibility of collapse. In the present embodiment, when the determination score is equal to or greater than the second threshold value, the second determination unit 72 outputs a flag indicating that there is a collapse (i = 1). On the other hand, when the determination score is less than the second threshold value, the second determination unit 72 outputs a flag indicating that there is no collapse (i = 0). Hereinafter, the flag (i = 1 or 0) related to the collapse determination output by the second determination unit 72 is referred to as the "second collapse determination flag". That is, the second collapse determination flag represents the result of the second determination and represents a value of 1 or 0. In addition, the second threshold value is stored in advance in the storage unit 90. The second determination unit 72 makes the second determination by appropriately referring to the second threshold value stored in the storage unit 90. The second determination unit 72 sends the second collapse determination flag, which is the result of the second determination, to the first collapse determination unit 75.

[0097] Here, Figure 7 shows Figure 5 the change over time in the determination score during the period shown and the change over time in the second collapse determination flag corresponding to the determination score during the same period. Figure 7 The graph shown is the result obtained through the analysis by the inventor. In Figure 7 (b) thereof, the moments when the operator of the incinerator 100 visually determines that there is a collapse of the refuse Fg in the combustion chamber R are indicated by circles (〇). In addition, the time corresponding to the intervals between the time scale lines (dashed lines) extending vertically in the graph shown in Figure 7 is equal in any interval. Based on Figure 7 the results shown, it has been found that the moments determined by the operator of the incinerator 100 as having a collapse are substantially the same as the moments when the second collapse determination flag appears (the moments when i = 1).

[0098] (Interference determination unit)

[0099] The interference determination unit 74 determines the presence or absence of interference based on a feature amount related to the luminance of one image (e.g., the most recent image) received from the acquisition unit 40. Hereinafter, the determination performed by the interference determination unit 74 is referred to as "interference determination". In the present embodiment, the interference determination unit 74 performs interference determination using the first infrared image that is the object of calculation by the first calculation unit 50 in the image received from the acquisition unit 40. In addition, when performing interference determination, the interference determination unit 74 may use an image different from the first infrared image in the image received from the acquisition unit 40. In addition, when performing interference determination, the interference determination unit 74 may use a plurality of images, and the first infrared image may be included in the plurality of images. In addition, when the interference determination unit 74 receives an infrared image from the acquisition unit 40, it may be converted from RAW data (16 bits) to, for example, BMP data (8 bits).

[0100] For example, the interference determination unit 74 calculates a feature amount related to the luminance of the entire first object region 42 in the first infrared image, a feature amount related to the luminance of each of the first region 42a to the ninth region 42i in the first object region 42, and a feature amount related to the luminance of a plurality of object regions that are a part of the first infrared image and different from the first object region 42. Hereinafter, the plurality of object regions different from the first object region 42 are referred to as "fourth object regions 45". As Figure 8 shown, the fourth object region 45 is divided into three parts in the first infrared image, for example, and the three parts of the fourth object region 45 are independent of each other. The "independence" mentioned here means a state in which the respective fourth object regions 45 are separated from each other in the first infrared image. One of the fourth object regions 45 ( Figure 8 45a in) is a part of the first infrared image above the first object region 42, and in this fourth object region 45, for example, the top of the furnace body 108 is reflected. In addition, one of the fourth object regions 45 ( Figure 8 45b in) is a part of the first infrared image to the right of the first object region 42, and in this fourth object region 45, for example, the side wall portion of the furnace body 108 is reflected. In addition, one of the fourth object regions 45 ( Figure 8 45c in) is a part of the first infrared image below the first object region 42, and in this fourth object region 45, for example, the garbage Fg undergoing afterburning in the afterburning region 132 is reflected. Therefore, the garbage Fg accumulated in the feeder 104 hardly enters the fourth object region 45. In the present embodiment, the plurality of fourth object regions 45 are set to be of different sizes from each other.

[0101] The interference determination unit 74 calculates two statistical quantities as feature quantities for the entire first target area 42, the first area 42a to the ninth area 42i, and the plurality of fourth target areas 45, respectively. In the present embodiment, the two statistical quantities are the average value of luminance and the standard deviation of luminance. Hereinafter, the feature quantity related to the luminance of the entire first target area 42 is referred to as "feature quantity A", the feature quantity related to the luminance in each of the first area 42a to the ninth area 42i is referred to as "feature quantity B", and the feature quantity related to the luminance of each of the plurality of fourth target areas 45 (45a, 45b, 45c) is referred to as "feature quantity C". In the present embodiment, the interference determination unit 74 determines whether the standard deviation of each calculated feature quantity (feature quantity A to feature quantity C) is included in a first range that is a predetermined numerical range. The interference determination unit 74 determines that the first infrared image is a high deviation image when, for example, one or more of the standard deviation of feature quantity A, the standard deviation of feature quantity B, and the standard deviation of feature quantity C are not included in the first range (outside the first range). On the other hand, the interference determination unit 74 determines that the first infrared image is a low deviation image when the standard deviation of feature quantity A, the standard deviation of feature quantity B, and the standard deviation of feature quantity C are all included in the first range. In addition, the interference determination unit 74 may also determine that the first infrared image is a high deviation image when two or more or all of the standard deviation of feature quantity A, the standard deviation of feature quantity B, and the standard deviation of feature quantity C are not included in the first range. In addition, the first range is stored in the storage unit 90 in advance. The interference determination unit 74 determines the first infrared image by appropriately referring to the first range stored in the storage unit 90.

[0102] The interference determination unit 74 makes a determination using a learned model that has been learned to output a determination result indicating the presence or absence of interference when the above feature quantities are input. Hereinafter, the learned model used by the interference determination unit 74 is referred to as "the learned model 92 for interference determination". The learned model 92 for interference determination is stored in the storage unit 90 in advance (refer to Figure 2 ). The interference determination unit 74 obtains the determination result output by inputting feature quantity A, feature quantity B, and feature quantity C to the learned model 92 for interference determination stored in the storage unit 90 as the result of interference determination. In the present embodiment, the learned model 92 for interference determination has a high deviation model and a low deviation model. When the interference determination unit 74 determines that the first infrared image is a high deviation image, it inputs only the feature quantities (feature quantity A, feature quantity B, and feature quantity C) to the high deviation model and obtains the determination result output from the high deviation model as the result of interference determination. On the other hand, when the interference determination unit 74 determines that the first infrared image is a low deviation image, it inputs only the feature quantities to the low deviation model and obtains the determination result output from the low deviation model as the result of interference determination.

[0103] In the present embodiment, the high-deviation model and the low-deviation model are supervised learning models such as SVM (Support Vector Machine), for example. Both the high-deviation model and the low-deviation model are generated (learned) by repeatedly performing a plurality of learning steps, which are input with the above-described feature amounts and taught the presence or absence of spalling corresponding to the image (correct answer data judged as correct by a person).

[0104] Here, Figure 9 shows an example (a) of a low-deviation image and an example (b) of a high-deviation image in the infrared image captured by the imaging device 2. Further, Figure 10 shows examples of infrared images in a form corresponding to the magnitude of the deviation from the brightness at a glance. As Figure 9 and Figure 10 show, it can be seen that the larger or smaller the deviation of the brightness as a feature amount is, the more difficult it is to grasp the situation inside the combustion chamber R. Based on Figure 9 and Figure 10 the feature amounts of a plurality of infrared images as shown are input to the learned model 92 for interference determination, and the presence or absence of interference corresponding to each infrared image (correct answer data) is taught, whereby the learned model 92 for interference determination is generated in advance. When the learned model 92 for interference determination that has completed learning is input with the feature amount based on a new infrared image, it outputs binary data (for example, 0 and 1) indicating the presence or absence of spalling corresponding to the infrared image. The interference determination unit 74 determines the presence or absence of interference by outputting a flag corresponding to the value of the binary data obtained from the learned model 92 for interference determination as an interference determination. Hereinafter, the flag related to the determination of interference output by the interference determination unit 74 is referred to as an "interference determination flag". Therefore, the interference determination flag indicates the result of the interference determination.

[0105] Here, Figure 11 shows the change over time series of the determination result related to the presence or absence of interference made by the operator of the incinerator 100 during a specific time period and the change over time series of the interference determination flag during the same time period. Figure 11 The graph shown in Figure 11 is the result obtained through the analysis by the inventor. Further, 0 ~T 9 is the same time period as Figure 5 and Figure 7 the time period T 0 ~T 9 shown in. Further, the time corresponding to the interval between the time scale lines (dashed lines) extending vertically in the graph shown in Figure 11 is equal in any interval. According to Figure 11The obtained results show that the time when an operator of the incineration device 100 visually observes the combustion chamber R and determines that there is interference is almost the same as the time when the interference determination flag appears.

[0106] (Third determination unit)

[0107] The third determination unit 73 makes a determination related to caving based on one or more images received from the acquisition unit 40. In the present embodiment, the third determination unit 73 receives a visible light image among the images received from the acquisition unit 40, and calculates a representative value of the luminance based on the received visible light image. The third determination unit 73 makes a determination related to caving based on the representative value of the luminance (third feature amount) received from the third calculation unit 60 and the representative value of the luminance (fourth feature amount) received from the fourth calculation unit 65. Hereinafter, the determination made by the third determination unit 73 is referred to as "third determination". The third determination unit 73 calculates, for each region (left region 43l, central region 43c, right region 43r), the feature change amount V2 (absolute value) generated between the monochromatic component image based on the first visible light image that is the object of the third determination and the monochromatic component images based on a plurality of second visible light images according to the following formula (iv). V2 = |C m (t) - (ΣD m (t u )) / u|...(iv)

[0108] In the above formula (iv), C m (t) is the representative value of the luminance (third feature amount) calculated by the third calculation unit 60 at time t. ΣD m (t u ) is the representative value of the luminance (fourth feature amount) calculated by the fourth calculation unit 65. t u is a plurality of times when each second visible light image (monochromatic component image) is acquired, and the time t u corresponds to each second visible light image one by one. u is the number of second visible light images (monochromatic component images), which is the number of frames acquired per unit time.

[0109] The third determination unit 73 makes a third determination as to whether the calculated feature change amount V2 for each area is equal to or greater than a predetermined third threshold value. When the feature change amount V2 in one or more areas among the calculated feature change amounts V2 for each area is equal to or greater than the third threshold value, the third determination unit 73 determines that there is a possibility of collapse. When the feature change amount V2 in all areas is less than the third threshold value, the third determination unit 73 determines that there is no possibility of collapse. In the present embodiment, when the feature change amount V2 is equal to or greater than the third threshold value, the third determination unit 73 outputs a flag indicating that there is a collapse (i = 1). On the other hand, when the feature change amount V2 is less than the third threshold value, the third determination unit 73 outputs a flag indicating that there is no collapse (i = 0). Hereinafter, the flag (i = 1 or 0) related to the determination of collapse output by the third determination unit 73 is referred to as the "third collapse determination flag". That is, the third collapse determination flag indicates the result of the third determination and represents a value of 1 or 0. In addition, the third threshold value is stored in advance in the storage unit 90. The third threshold value may also be a value that varies for each area. The third determination unit 73 makes the third determination by appropriately referring to the third threshold value stored in the storage unit 90. The third determination unit 73 sends the third collapse determination flag, which is the result of the third determination, to the first collapse determination unit 75.

[0110] Here, Figure 12 The time-series changes of the feature change amount V2 of each area (left area 43l, central area 43c, and right area 43r) in a specific time period and the time-series changes of the third collapse determination flag corresponding to the feature change amount V2 in the same time period are shown. Figure 12 The graph shown in is the result obtained through the analysis by the inventor. In Figure 12 (d) thereof, the moments when the operator of the incineration apparatus 100 visually determines that there is a collapse of the garbage Fg in the combustion chamber R are indicated by circles (〇). In addition, the time corresponding to the interval between the time scale lines (dashed lines) extending vertically in the graph shown in Figure 12 is equal in any interval. According to Figure 12 the results shown, it is found that the moments determined by the operator of the incineration apparatus 100 as having a collapse are roughly the same as the moments when the third collapse determination flag appears (the moments when i = 1).

[0111] (First Collapse Determination Unit)

[0112] The first collapse determination unit 75 determines the scale of the collapse based on the result of the first determination received from the first determination unit 71, the result of the second determination received from the second determination unit 72, and the result of the third determination received from the third determination unit 73. In the present embodiment, the first collapse determination unit 75 determines whether there is a collapse of a second scale that is larger than the collapse of the first scale based on the first collapse determination flag, the second collapse determination flag, and the third collapse determination flag. Specifically, when the sum of the values of the first collapse flag, the second collapse flag, and the third collapse flag is equal to or greater than the determination threshold, the first collapse determination unit 75 determines that there is a collapse of the second scale. That is, the first collapse determination unit 75 detects a collapse of the second scale. When the first collapse determination unit 75 detects a collapse of the second scale, it outputs a flag indicating that there is a collapse. On the other hand, when the sum of the values of the first collapse determination flag, the second collapse determination flag, and the third collapse determination flag is less than the determination threshold, the first collapse determination unit 75 determines that there is no collapse of the second scale. That is, the first collapse determination unit 75 detects that there is no collapse of the second scale. When the first collapse determination unit 75 detects that there is no collapse of the second scale, it outputs a flag indicating that there is no collapse of the second scale. Hereinafter, the flag output by the first collapse determination unit 75 is referred to as the "second-scale collapse detection flag". In addition, the determination threshold is stored in advance in the storage unit 90. In the present embodiment, the determination threshold is an integer, and for example, 2 is used. That is, in the present embodiment, the first collapse determination unit 75 determines that there is a collapse of the second scale when more than two-thirds of the determination results among the results of the first determination, the second determination, and the third determination indicate the possibility of a collapse. The first collapse determination unit 75 determines whether there is a collapse of the second scale by appropriately referring to the determination threshold stored in the storage unit 90. The first collapse determination unit 75 sends the second-scale collapse detection flag to the second collapse determination unit 76 and the control unit 80.

[0113] (Second collapse determination unit)

[0114] When the first caving determination unit 75 determines that there is no caving of the second scale, the second caving determination unit 76 determines whether there is caving of the first scale based on the change in the brightness of a plurality of images received from the acquisition unit 40. In the present embodiment, the second caving determination unit 76 determines whether there is caving of the first scale based on the first caving determination flag received from the first determination unit 71. Specifically, when the first caving determination flag indicates that there is caving (i = 1), the second caving determination unit 76 determines that there is caving of the first scale. That is, the second caving determination unit 76 detects caving of the first scale. When the second caving determination unit 76 detects caving of the first scale, it outputs a flag indicating that there is caving. On the other hand, when the first caving determination flag indicates that there is no caving (i = 0), the second caving determination unit 76 determines that there is no caving. That is, the second caving determination unit 76 detects that there is no caving. When the second caving determination unit 76 detects caving of the first scale, it outputs a flag indicating that there is no caving. Hereinafter, the flag output by the second caving determination unit 76 is referred to as the "first-scale caving detection flag". In addition, hereinafter, when not distinguishing between the "first-scale caving detection flag" and the above-mentioned "second-scale caving detection flag", it is simply referred to as the "caving detection flag". The second caving determination unit 76 sends the first caving detection flag to the control unit 80.

[0115] (Structure of the control unit)

[0116] The control unit 80 controls a plurality of controlled devices S based on the caving detection flags received from the first caving determination unit 75 and the second caving determination unit 76 (refer to Figure 1 and Figure 2 ). In the present embodiment, when the control unit 80 receives the second-scale caving detection flag indicating that there is caving from the first caving determination unit 75, for example, it controls one or more of the plurality of controlled devices S to reduce the concentration of unburned gas in the combustion chamber R. On the other hand, when the control unit 80 receives the first-scale caving detection flag from the second caving determination unit 76, for example, it controls one or more of the plurality of controlled devices S to cause the controlled devices S to operate at rated conditions. The control unit 80 sends signals indicating, for example, the increase or decrease in the moving speed of the push-out arm 124, the moving speed of the grate bars 126, the increase or decrease in the rotational speed of the blower 138, the valve opening degree of the first flow rate adjustment valve 140, and the valve opening degree of the second flow rate adjustment valve 142 to each controlled device S. In addition, when the control unit 80 receives the first-scale caving detection flag from the second caving determination unit 76, it may also control one or more of the plurality of controlled devices S so that the concentration of unburned gas in the combustion chamber R is reduced to a lesser extent than when it receives the second-scale caving detection flag.

[0117] (Operation of the information processing device)

[0118] Next, with reference to Figure 13 an example of the operation of the information processing apparatus 4 in the present embodiment will be described. However, the order of the processes described below is not limited to the following examples, and may be replaced as appropriate.

[0119] The acquisition unit 40 acquires an infrared image from the imaging device 2 (step S1). In addition, the acquisition unit 40 acquires a visible light image from the imaging device 2 (step S11). After the process of step S1, the first calculation unit 50 calculates a representative value of the luminance based on the first infrared image acquired in step S1. In addition, the second calculation unit 55 calculates a representative value of the luminance based on the second infrared image acquired in step S1 (step S2). Next, the interference determination unit 74 performs interference determination based on the first infrared image acquired in step S1 (step S3). When it is determined by the interference determination unit 74 that there is interference (step S3: Yes), the process returns to step S1. On the other hand, when it is determined by the interference determination unit 74 that there is no interference (step S3: No), the first determination unit 71 makes a first determination based on the representative value of the luminance calculated in step S2 (step S4), and the second determination unit 72 makes a second determination based on the representative value of the luminance calculated in step S2 (step S5).

[0120] After the process of step S11, the third calculation unit 60 calculates a representative value of the luminance based on the first visible light image acquired in step S11. In addition, the fourth calculation unit 65 calculates a representative value of the luminance based on the second visible light image acquired in step S1 (step S12). Next, the third determination unit 73 makes a third determination based on the representative value of the luminance calculated in step S12 (step S13).

[0121] After the processes of steps S4, S5, and S13, the first collapse determination unit 75 determines whether there is a second-scale collapse based on the determination results of step S4, the determination result of step S5, and the determination result of step S6 (step S6). That is, in step S6, the first collapse determination unit 75 determines whether the total value of the first collapse determination flag, the second collapse determination flag, and the third collapse determination flag is equal to or greater than the determination threshold value. When the first collapse determination unit 75 determines that there is a second-scale collapse (step S6: Yes), it detects a second-scale collapse (step S7). When the process of step S7 ends, the process returns to step S1.

[0122] On the other hand, when the first collapse determination unit 75 determines that there is no collapse of the second scale (step S6: No), the second collapse determination unit 76 determines whether there is a change in the representative value of the luminance (step S8). That is, in step S8, the second collapse determination unit 76 determines whether there is a collapse of the first scale based on the first collapse determination flag. When the second collapse determination unit 76 determines that there is a collapse of the first scale (step S8: Yes), it detects the collapse of the first scale (step S9). When the process of step S9 ends, the process returns to step S1. On the other hand, when the second collapse determination unit 76 determines that there is no collapse of the first scale (step S8: No), it detects that there is no collapse (step S10). When the process of step S10 ends, the process returns to step S1.

[0123] The operation of the information processing device 4 described above is repeatedly executed during the operation stage of the incineration device 100.

[0124] (Function, effect)

[0125] Complex phenomena occur in the combustion chamber R, so it is sometimes difficult to appropriately detect the collapse of the refuse Fg. For example, as one of the reasons, it can be cited that in the combustion chamber R, ash 135 and the like fly at an unwanted timing, and the flying ash 135 momentarily enters the image.

[0126] In the present embodiment, it is determined whether the refuse Fg has collapsed based on the representative value of the luminance based on the first infrared image and the representative value of the luminance based on a plurality of second infrared images taken within the time taken for one collapse of the refuse Fg among a plurality of infrared images in the time series obtained before the first infrared image. Therefore, compared with the case of determining whether there is a collapse based on the luminance of each image obtained at two timings, for example, it is possible to determine whether the refuse Fg has collapsed with higher accuracy. That is, the detection accuracy related to the collapse of the refuse Fg can be improved. As a result, the combustion state of the refuse Fg in the combustion chamber R can be accurately grasped.

[0127] Figure 14 It is a diagram showing an example of the result in the time series of the collapse detection flag output by the first collapse determination unit 75 and the second collapse determination unit 76 of the present embodiment. That is, in Figure 14 it shows the final determination result (detection result) of the information processing device 4 in the present embodiment. Figure 14 The graph shown in is the result obtained through the analysis by the inventors. In Figure 14 it, the moment when the operator of the incineration device 100 visually observes the combustion chamber R and determines that there is a collapse of the refuse Fg of the first scale is indicated by a triangle (△), and the moment when it is determined that there is a collapse of the refuse Fg of the second scale is indicated by a circle (〇). In addition, in Figure 14The time scale lines (single dotted lines) extending vertically in the chart shown in are spaced from each other by an equal time interval in any interval. According to Figure 14 the results shown in , although it is possible to see the time (moment T -18 and moment T -17 between) when the avalanche that might be over-detected as having garbage Fg by the information processing device 4 occurs, it is possible to grasp that the moment determined by the operator of the incineration device 100 as having an avalanche almost coincides with the moment when the avalanche determination flag appears.

[0128] <Second Embodiment of Incineration Device>

[0129] Next, a second embodiment of the incineration device 100 of the present disclosure will be described. In addition, in the second embodiment described below, for the structures shared with the above-described first embodiment, the same reference numerals are given in the drawings and their descriptions are omitted. In the second embodiment, the structure of the second avalanche determination unit 76 of the avalanche detection unit 41 is different from that of the second avalanche determination unit 76 described in the above first embodiment.

[0130] In the present embodiment, the second avalanche determination unit 76 calculates the similarity between the image acquired by the acquisition unit 40 and one or more images acquired in the past by the acquisition unit 40, and determines that there is no avalanche when the calculated similarity does not satisfy a predetermined condition. Hereinafter, the determination of whether there is an avalanche by the second avalanche determination unit 76 will be described by taking as an example the case where the second determination unit 72 calculates the similarity with respect to the first infrared image at time t and a plurality of infrared images acquired in the past before the first infrared image.

[0131] As Figure 15 shown, the second avalanche determination unit 76 binarizes (converts to 0 / 1) the brightness values included in a specific object area (b), which is a part of the infrared image (a) received from the acquisition unit 40, into data of 0 and 1. Hereinafter, the object area that is the object of binarization by the second avalanche determination unit 76 will be referred to as the "fifth object area 46". The fifth object area 46 is, for example, an area in the upper half of the infrared image where a part (most) of the garbage Fg accumulated in the feeder 104 is reflected. The second avalanche determination unit 76 binarizes the brightness values included in the fifth object area 46 based on a predetermined brightness threshold value. In addition, the brightness threshold value is stored in advance in the storage unit 90. The second avalanche determination unit 76 binarizes the fifth object area 46 by appropriately referring to the brightness threshold value stored in the storage unit 90. Hereinafter, the fifth object area 46 binarized by the second avalanche determination unit 76 will be referred to as the "binarized data 46'".

[0132] Figure 16 is a diagram for conceptually explaining the calculation method of the similarity calculated by the second avalanche determination unit 76. AsFigure 16 As shown, the second caving determination unit 76 obtains the difference between the binarized data 46' of the first infrared image based on the time t and the binarized data 46' of a plurality of infrared images respectively obtained in the past before the first infrared image ( Figure 16 X1, X2, …, Xy-1, Xy shown in Figure 16 and calculates the average value of the obtained plurality of differences (

[0133] ΣX / y shown in

[0134] as the similarity. The binarized data 46' respectively based on the plurality of infrared images mentioned here respectively refers to, for example, the binarized data 46' of the infrared image obtained at time t-1, the binarized data 46' of the infrared image obtained at time t-2, …, the binarized data 46' of the infrared image obtained at time t-(y-1), and the binarized data 46' of the infrared image obtained at time t-y. y is an integer, for example, a value of 2 or more (such as 5) is adopted. Figure 17 Figure 13

[0135]

[0135] The second caving determination unit 76 determines whether the calculated similarity satisfies a predetermined condition. In the present embodiment, the second caving determination unit 76 determines whether the similarity is equal to or greater than a predetermined similarity threshold. At this time, the second caving determination unit 76 determines that the predetermined condition is satisfied when the similarity is equal to or greater than the similarity threshold, and determines that the predetermined condition is not satisfied when the similarity is less than the similarity threshold. In addition, the similarity threshold is stored in the storage unit 90 in advance. The second caving determination unit 76 determines whether the calculated similarity satisfies the predetermined condition by timely referring to the similarity threshold stored in the storage unit 90. When the similarity does not satisfy the predetermined condition, the second caving determination unit 76 determines that there is a caving of the first scale. That is, the second caving determination unit 76 detects a caving of the first scale. When the second caving determination unit 76 detects a caving of the first scale, it outputs a first scale caving detection flag indicating that there is a caving. On the other hand, when the similarity satisfies the predetermined condition, the second caving determination unit 76 determines that there is no caving. That is, the second caving determination unit 76 detects that there is no caving. When the second caving determination unit 76 detects a caving of the first scale, it outputs a caving detection flag indicating that there is no caving. Next, with reference to Figure 17 an example of the operation of the information processing device 4 will be described. However, the order of the processes described below is not limited to the following examples and can be appropriately replaced. In addition, the description of the parts that repeat the operation of the information processing device 4 described using Figure 13 is omitted.

[0135] When the second collapse determination unit 76 determines that there is a change in the representative value of the luminance (step S8: Yes), it calculates the similarity and determines whether the calculated similarity satisfies a predetermined condition (step S20). On the other hand, when the second collapse determination unit 76 determines that there is no change in the representative value of the luminance (step S8: No), it detects that there is no collapse (step S10). When the similarity does not satisfy the predetermined condition (step S20: No), the second collapse determination unit 76 detects a collapse of the first scale (step S9). On the other hand, when the similarity satisfies the predetermined condition (step S20: Yes), the second collapse determination unit 76 executes the process of step S10.

[0136] Figure 18 FIG. is an example of the result of the time series of the collapse detection flags output by the above-described first collapse determination unit 75 and second collapse determination unit 76. That is, in Figure 18 the final determination result (detection result) of the information processing device 4 is shown. Figure 18 The graph shown in is the result obtained through the analysis by the inventors. In Figure 18 the moments when it is determined that there is a collapse of the garbage Fg of the first scale by visually observing the combustion chamber R by the operator of the incineration device 100 are indicated by triangles (△), and the moments when it is determined that there is a collapse of the garbage Fg of the second scale are indicated by circles (〇). In addition, the time corresponding to the intervals between the time scale lines (dashed lines) extending vertically in the graph shown in is equal in any interval. In the result shown in Figure 18 compared with the result shown in Figure 18 there is no time when the information processing device 4 may be over-detected as having a collapse of the garbage Fg ( Figure 14 the moment T in Figure 14 and the moment T -18 and the moment T -17 between). In addition, it is grasped that the moment when the operator of the incineration device 100 determines that there is a collapse coincides with the moment when the collapse determination flag appears.

[0137] <Third Embodiment of Incineration Device>

[0138] Next, a third embodiment of the incineration device 100 of the present disclosure will be described. In addition, in the third embodiment described below, for the structures common to the above-described first embodiment, the same reference numerals are given in the drawings and the description thereof is omitted. In the third embodiment, the structures of the collapse detection unit 41 and the storage unit 90 are different from those of the first embodiment.

[0139] (Structure of Collapse Detection Unit)

[0140] As Figure 19As shown, the caving detection unit 41 in this embodiment includes: a first calculation unit 50, a second calculation unit 55, a fifth calculation unit 66, a sixth calculation unit 67, and a determination unit 70. The determination unit 70 includes: an interference determination unit 74, a fourth determination unit 77, a fifth determination unit 78, a third caving determination unit 79, and a fourth caving determination unit 81.

[0141] (First calculation unit)

[0142] As Figure 20 shown, in this embodiment, the first object area 42 that is the object of calculation by the first calculation unit 50 is divided into 24 small areas with equal areas in a 6x4 matrix shape. In addition, the first object area 42 is not limited to the case of being equally divided into 24 small areas in a matrix shape.

[0143] The first calculation unit 50 calculates, for example, the average value of the brightness of each small area in the first object area 42 in the first infrared image. In addition, the representative value calculated by the first calculation unit 50 is not limited to the average value of the brightness of each small area, and may be, for example, a statistic such as a median value.

[0144] (Fifth calculation unit)

[0145] When the interference determination unit 74 determines that there is no interference, the fifth calculation unit 66 calculates a fifth feature quantity based on the representative value of the brightness (first feature quantity) received from the first calculation unit 50 and the representative value of the brightness (second feature quantity) received from the second calculation unit 55. In this embodiment, the fifth feature quantity is the sum of the brightness change amounts obtained by performing a time difference on the average value of the brightness of each of the 24 small areas in the first object area 42 in the first infrared image. In addition, the sum calculated by the fifth calculation unit 66 is not limited to the sum of the brightness change amounts of each small area, and may be, for example, the sum of statistics such as a median value. The fifth calculation unit 66 inputs the sum of the brightness change amounts of each small area in the first object area 42 in the first infrared image to the third caving determination unit 79.

[0146] (Sixth calculation unit)

[0147] The sixth calculation unit 67 performs unsupervised learning using the algorithm of an auto encoder (Auto Encoder), and calculates a sixth feature quantity using the information after dimensionality reduction placed in the hidden layer of the learned auto encoder. In this embodiment, the auto encoder 96 is pre-stored in the storage unit 90 (refer to Figure 19). The sixth feature quantity is infrared image information obtained by encoding a plurality of consecutive infrared images acquired by the imaging device 2 with an autoencoder 96 and having the dimension reduced. The imaging device 2 acquires 1 frame of infrared image every 0.1 seconds. The sixth calculation unit 67 inputs a plurality of consecutive infrared images acquired by the imaging device 2 at intervals of 0.1 seconds to the autoencoder 96. In the autoencoder 96, the plurality of infrared images are compressed (encoded) as they flow from the input layer to the hidden layer, and the dimension is reduced to 512 dimensions. Then, the infrared image with the reduced dimension is restored (decoded) to the original information as it flows from the hidden layer to the output layer. If the infrared image placed in the input layer can be restored from the infrared image flowing from the hidden layer to the output layer, the learning data of this infrared image does not contain abnormal data such as interference images, and it is regarded as correct learning. When correct learning is performed, the infrared image information with the reduced dimension reduces the noise in the image. In addition, the interval at which the imaging device 2 acquires infrared images is not limited to 0.1 seconds. In addition, the dimension reduction implemented in the autoencoder 96 is not limited to 512 dimensions.

[0148] When the third collapse determination unit 79 determines that there is a collapse in the furnace as described later, the sixth calculation unit 67 aggregates and packages the noise-removed infrared image information of 40 consecutive frames in total, which is obtained 2 seconds before and after the time point determined by a person as the time of garbage collapse as the base point, from the plurality of infrared image information with the dimension reduced and the noise in the image removed and acquired at intervals of 0.1 seconds, and inputs it to the long short-term memory (LSTM: Long Short-Term Memory (long short-term memory network)) network included in the fourth collapse determination unit 81. The long short-term memory network is a type of RNN that performs learning and prediction (regression, classification) of time series data. In the fourth collapse determination unit 81, a learned model is used for determination. The learned model is learned in such a way that when the packaged infrared image information is input to the long short-term memory network, it outputs a determination result related to the possibility of collapse. Hereinafter, the learned model used in the determination performed by the fourth collapse determination unit 81 is referred to as the "learned model 97 for collapse determination". The learned model 97 for collapse determination is pre-stored in the storage unit 90 (refer to Figure 19 ). The number of frames of the packaged information is not limited to 40. It can be appropriately increased or decreased considering the characteristics of the incinerator, etc., according to how much time it takes to observe a series of phenomena such as garbage peeling off from the garbage surface in the furnace, the garbage reaching the furnace bed, the garbage flying in the furnace, and the garbage flying in the furnace falling. In addition, since there is uncertainty in the human judgment of the occurrence of garbage collapse, the infrared image information of 5 frames including the time point determined by a person as the time of garbage collapse can also be used as the learning data for the occurrence of collapse.

[0149] (Fourth Determination Unit)

[0150] Based on one or more infrared images received from the acquisition unit 40, the fourth determination unit 77 makes a determination related to caving. In the present embodiment, the fourth determination unit 77 uses one first infrared image (the latest infrared image) that is the object of calculation by the first calculation unit 50 among the images received from the acquisition unit 40 to make a determination related to caving. In addition, when making a determination of caving, the fourth determination unit 77 may also use an image different from the first infrared image among the images received from the acquisition unit 40. In addition, when making a determination of caving, the fourth determination unit 77 may use multiple images, and the first infrared image may also be included in the multiple images.

[0151] The fourth determination unit 77 makes a determination using a learned model that has been learned in such a way that when the first infrared image is input, a determination result related to the possibility of caving occurring is output. Hereinafter, the determination made by the fourth determination unit 77 is referred to as "fourth determination", and the learned model used by the fourth determination unit 77 in the fourth determination is referred to as "fourth determination learned model 94". The fourth determination learned model 94 is pre-stored in the storage unit 90 (refer to Figure 19 ). The fourth determination unit 77 obtains the determination result output by inputting the first infrared image received from the acquisition unit 40 to the fourth determination learned model 94 stored in the storage unit 90 as the result of the fourth determination. The fourth determination learned model 94 is, for example, a deep learning model (supervised learning model) such as a convolutional neural network (CNN). The fourth determination learned model 94 is generated (learned) by repeatedly performing a learning process in which an infrared image captured by the imaging device 2 is input and the presence or absence of caving corresponding to the infrared image (correct solution data judged to be correct by a person) is taught. In addition, the fourth determination learned model 94 may use a recurrent neural network (RNN) or the like instead of the CNN.

[0152] Figure 21Examples of images (a, d) showing that the waste Fg is flying (there is a collapse) in the infrared image captured by the imaging device 2, examples of images (b, e) showing that the ash is flying (there is no collapse) without the influence of a collapse, and examples of images (c, f, including the retention of water vapor) showing other states (no collapse). In the present embodiment, when the infrared image of the object region that is a part of the first infrared image is input to the fourth determination learned model 94, the fourth determination unit 77 classifies whether there is a collapse in the infrared image captured by the imaging device 2 based on the above three classified image examples. The fourth determination learned model 94 is pre-generated by teaching the presence or absence of a collapse (correct data) corresponding to the infrared image. When the learned fourth determination learned model 94 is input with a new infrared image, a value related to the possibility of a collapse occurring corresponding to the infrared image is output as a "determination score".

[0153] The fourth determination unit 77 determines whether the determination score obtained from the fourth determination learned model 94 is equal to or higher than a predetermined fourth threshold value (fourth determination). When the obtained determination score is equal to or higher than the fourth threshold value, the fourth determination unit 77 determines that there is a possibility of a collapse, and when the determination score is less than the fourth threshold value, the fourth determination unit 77 determines that there is no possibility of a collapse. In the present embodiment, when the determination score is equal to or higher than the fourth threshold value, the fourth determination unit 77 outputs a flag indicating that there is a collapse (i = 1). On the other hand, when the determination score is less than the fourth threshold value, the fourth determination unit 77 outputs a flag indicating that there is no collapse (i = 0). Hereinafter, the flag (i = 1 or 0) related to the collapse determination output by the fourth determination unit 77 is referred to as a "fourth collapse determination flag". That is, the fourth collapse determination flag indicates the result of the fourth determination and represents a value of 1 or 0. The fourth threshold value is pre-stored in the storage unit 90. The fourth determination unit 77 performs the fourth determination by appropriately referring to the fourth threshold value stored in the storage unit 90. The fourth determination unit 77 inputs the fourth collapse determination flag, which is the result of the fourth determination, to the third collapse determination unit 79.

[0154] (Fifth determination unit)

[0155] The fifth determination unit 78 makes a determination related to a collapse based on the visible light image received from the acquisition unit 40. In the present embodiment, the fifth determination unit 78 uses the first visible light image (the latest visible light image) in the image received from the acquisition unit 40 to make a determination related to a collapse. In addition, when making a determination of a collapse, the fifth determination unit 78 may also use one image different from the first visible light image in the image received from the acquisition unit 40. In addition, when making a determination of a collapse, the fifth determination unit 78 may use a plurality of images, or may include the first visible light image in the plurality of images.

[0156] The fifth determination unit 78 makes a determination using a learned model that has been learned in such a way that when a first visible light image is input, it outputs a determination result related to the possibility of occurrence of a collapse. Hereinafter, the determination by the fifth determination unit 78 is referred to as "fifth determination", and the learned model used by the fifth determination unit 78 in the fifth determination is referred to as "fifth determination learned model 95". The fifth determination learned model 95 is pre-stored in the storage unit 90 (see Figure 19 ). The fifth determination unit 78 obtains the determination result output by inputting the first visible light image received from the acquisition unit 40 to the fifth determination learned model 95 stored in the storage unit 90 as the result of the fifth determination. The fifth determination learned model 95 is, for example, a deep learning model (supervised learning model) such as a convolutional neural network (CNN). The fifth determination learned model 95 is generated (learned) by repeatedly performing a plurality of learning steps, in which the visible light image captured by the imaging device 2 is input, and the presence or absence of a collapse corresponding to the visible light image (correct answer data judged to be correct by a person) is taught. In addition, the fifth determination learned model 95 may use a recurrent neural network (RNN) or the like instead of the CNN.

[0157] The fifth determination unit 78 classifies the first visible light image based on an image example of garbage covering a flame during a large-scale collapse and an image example representing a state other than that (no collapse). That is, the visible light image of the object area that is part of the first visible light image is input to the fifth determination learned model 95, and based on the above two image examples, the presence or absence of a collapse corresponding to the first visible light image is classified. The fifth determination learned model 95 is pre-generated by teaching the presence or absence of a collapse (correct answer data) corresponding to the visible light image. When the learned fifth determination learned model 95 is input with a new visible light image, it outputs a value related to the possibility of occurrence of a collapse corresponding to the visible light image as a "determination score".

[0158] The fifth determination unit 78 determines whether the determination score obtained from the fifth learned determination model 95 is equal to or greater than a predetermined fifth threshold (fifth determination). When the obtained determination score is equal to or greater than the fifth threshold, the fifth determination unit 78 determines that there is a possibility of caving. When the determination score is less than the fifth threshold, the fifth determination unit 78 determines that there is no possibility of caving. In the present embodiment, when the determination score is equal to or greater than the fifth threshold, the fifth determination unit 78 outputs a flag indicating caving (i = 1). On the other hand, when the determination score is less than the fifth threshold, the fifth determination unit 78 outputs a flag indicating no caving (i = 0). Hereinafter, the flag related to the caving determination output by the fifth determination unit 78 (i = 1 or 0) is referred to as the "fifth caving determination flag". The fifth threshold is stored in the storage unit 90 in advance. The fifth determination unit 78 performs the fifth determination by appropriately referring to the fifth threshold stored in the storage unit 90. The fifth determination unit 78 inputs the fifth caving determination flag, which is the result of the fifth determination, to the third caving determination unit 79.

[0159] (Third caving determination unit)

[0160] The third caving determination unit 79 determines whether there is caving of the garbage in the furnace based on the sum of the luminance change amounts of the 24 small regions in the first object region 42 in the first infrared image calculated by the fifth calculation unit 66, the result of the fourth determination in the fourth determination unit 77, that is, the fourth caving determination flag, and the result of the fifth determination in the fifth determination unit 78, that is, the fifth caving determination flag. Specifically, three patterns are prepared by combining the sum of the luminance change amounts in the first object region 42, the fourth caving determination flag, and the fifth caving determination flag. As Figure 22 shown, the first pattern is when the sum of the respective luminance change amounts in the first object region 42 is 22 or more or - (negative) 26 or less, and the fourth caving determination flag input from the fourth determination unit 77 is a flag indicating caving (i = 1). The second pattern is when the sum of the luminance change amounts of the small regions in the first object region 42 is 20 or more or - (negative) 13 or less, and the fifth caving determination flag input from the fifth determination unit 78 is a flag indicating caving (i = 1). The third pattern is when the sum of the luminance change amounts of the small regions in the first object region 42 is 10 or more or - (negative) 13 or less, and both the fourth caving determination flag input from the fourth determination unit 77 and the fifth caving determination flag input from the fifth determination unit 78 are flags indicating caving (i = 1).

[0161] When the sum of the luminance change amounts of each of the 24 small regions in the first object region 42 in the first infrared image calculated by the fifth calculation unit 66, the fourth collapse determination flag input from the fourth determination unit 77, and the fifth collapse determination flag input from the fifth determination unit 78 conform to any one of the above first to third patterns, it is determined that there is a collapse in the furnace. When they do not conform to any one of the above first to third patterns, it is determined that there is no collapse in the furnace. In addition, the threshold value of the sum of the luminance change amounts of the infrared images included in the above first to third patterns may also vary depending on the type of the selected image. That is, when the determination result based on deep learning of the visible light image indicates a collapse, it can be regarded that a large-scale garbage collapse has occurred in the furnace. Therefore, the range of the threshold value of the sum of the luminance change amounts of the infrared images included in the second pattern may also be smaller than the range of the threshold value of the sum of the luminance change amounts of the infrared images included in the first pattern. In addition, when the determination results based on deep learning of the infrared image and the visible light image both indicate a collapse, it can be regarded that the accuracy of the garbage collapse occurring in the furnace is relatively high. Therefore, the range of the threshold value of the sum of the luminance change amounts of the infrared images included in the third pattern may also be smaller than the range of the threshold value of the sum of the luminance change amounts of the infrared images included in the first and second patterns.

[0162] (Fourth Collapse Determination Unit)

[0163] The fourth collapse determination unit 81 includes a long short-term memory network which is a kind of RNN for learning or predicting (regressing, classifying) time series data. The fourth collapse determination unit 81 uses the learned collapse determination completed model 97 which is learned in such a way that when a packet of infrared image information of 40 consecutive frames with noise removed from the image is input to the long short-term memory network from the sixth calculation unit 67, a determination result related to the possibility of a collapse is output, to determine whether there is a garbage collapse in the furnace. The fourth collapse determination unit 81 inputs a packet of infrared image information of 40 consecutive frames with noise removed from the image input from the sixth calculation unit 67 to the collapse determination completed model 97 stored in the storage unit 90, and obtains a determination result based on the long short-term memory network.

[0164] When a packet of infrared image information of 40 consecutive frames with noise removed from the image is input from the sixth calculation unit 67 to the long short-term memory network, the fourth caving determination unit 81 causes the long short-term memory network to learn this packet and determines the state inside the furnace based on the state transition of the 40-frame amount of this infrared image. That is, when a packet of infrared image information of 40 consecutive frames is input to the learned model 97 for caving determination, the presence or absence of caving (correct answer data) corresponding to this infrared image information is taught based on the state inside the furnace over time, and thus the learned model 97 for caving determination is generated in advance. When the learned model 97 for caving determination that has completed learning is input with a packet of infrared image information, a value related to the possibility of caving corresponding to the 40-frame amount of infrared image information included in this packet is output as a "determination score".

[0165] The fourth caving determination unit 81 determines whether the determination score obtained from the learned model 97 for caving determination is equal to or greater than a predetermined sixth threshold (sixth determination). When the obtained determination score is equal to or greater than the sixth threshold, the fourth caving determination unit 81 determines that there is a possibility of caving, and when the determination score is less than the sixth threshold, the fourth caving determination unit 81 determines that there is no possibility of caving. In the present embodiment, when the determination score is equal to or greater than the sixth threshold, the fourth caving determination unit 81 outputs a flag indicating that there is caving (i = 1). On the other hand, when the determination score is less than the sixth threshold, the fourth caving determination unit 81 outputs a flag indicating that there is no caving (i = 0). When the fourth caving determination unit 81 outputs a flag (i = 1), the determination unit 70 finally determines that there is caving inside the furnace, and when the fourth caving determination unit 81 outputs a flag (i = 0), the determination unit 70 finally determines that there is no caving inside the furnace.

[0166] (Structure of the control unit)

[0167] The control unit 80 controls a plurality of controlled devices S based on the caving detection flag received from the fourth caving determination unit 81 (refer to Figure 19 ). In the present embodiment, when the control unit 80 receives a caving detection flag indicating that there is caving from the fourth caving determination unit 81, the control unit 80 controls one or more of the plurality of controlled devices S so that the controlled devices S perform rated operation. The control unit 80 sends signals such as signals indicating the increase or decrease of the moving speed of the pushing arm 124, the moving speed of the grate bars 126, the rotational speed increase or decrease of the blower 138, the valve opening increase or decrease of the first flow rate adjustment valve 140, and the valve opening increase or decrease of the second flow rate adjustment valve 142 to each controlled device S. In addition, the control unit 80 may also control one or more of the plurality of controlled devices S to reduce the concentration of unburned gas in the combustion chamber R.

[0168] (Operation of the information processing device)

[0169] Next, with reference to Figure 23 an example of the operation of the information processing apparatus 4 in the present embodiment will be described. However, the order of the processes described below is not limited to the following examples and may be appropriately replaced.

[0170] The acquisition unit 40 acquires an infrared image from the imaging device 2 (step S1). In addition, the acquisition unit 40 acquires a visible light image from the imaging device 2 (step S11). After the process of step S1, the first calculation unit 50 calculates a representative value of the luminance based on the first infrared image acquired in step S1. In addition, the second calculation unit 55 calculates a representative value of the luminance based on the second infrared image acquired in step S1 (step S2). Next, the interference determination unit 74 performs interference determination based on the first infrared image acquired in step S1 (step S3). When it is determined by the interference determination unit 74 that there is interference (step S3: Yes), the process returns to step S1. On the other hand, when it is determined by the interference determination unit 74 that there is no interference (step S3: No), the fifth calculation unit 66 calculates a fifth feature amount based on the representative value of the luminance (first feature amount) received from the first calculation unit 50 and the representative value of the luminance (second feature amount) received from the second calculation unit 55 (step S21). The fourth determination unit 77 performs a determination related to caving based on one or more infrared images received from the acquisition unit 40 (step S22). In addition, the fifth determination unit 78 performs a determination related to caving based on the visible light image received from the acquisition unit 40 (step S23).

[0171] After the processes of steps S21, S22, and S23 described above, the third caving determination unit 79 determines whether there is caving of the garbage in the furnace based on the sum of the luminance change amounts of the small regions in the first object region 42 in the first infrared image calculated in step S21, the determination result of step S22, and the determination result of step S23 (step S24). When it is determined by the third caving determination unit 79 that there is caving (step S24: i = 1), the sixth calculation unit 67 aggregates and packs 40 frames of noise-removed infrared image information that is continuous at 0.1-second intervals and has had the noise removed from the image, with a total of 4 seconds before and after the time point determined by a human to be when garbage caving occurred, and inputs it to the long short-term memory network included in the fourth caving determination unit 81 (step S25). When it is determined by the third caving determination unit 79 that there is no caving (step S24: i = 0), it is detected that there is no caving of the garbage in the furnace (step 28). When the process of step S28 ends, the process returns to step S1.

[0172] After the processing in step S25, when a packet of infrared image information of 40 consecutive frames with noise removed from the image is input from the sixth calculation unit 67 to the long short-term memory network, the fourth caving determination unit 81 determines whether there is caving of refuse in the furnace using the learned caving determination model 97 (step S26). When it is determined by the fourth caving determination unit 81 that there is caving (step S26: i = 1), caving of refuse in the furnace is detected (step S27). When the processing in step S27 ends, the process returns to the processing of step S1. When it is determined by the fourth caving determination unit 81 that there is no caving (step S26: i = 0), it is detected that there is no caving (step S28). When the processing in step S28 ends, the process returns to the processing of step S1.

[0173] The operations of the information processing apparatus 4 described above are repeatedly executed during the operation stage of the incineration apparatus 100.

[0174] (Function, effect)

[0175] Even if an image that cannot correctly represent the change in the state inside the furnace during caving, such as the state where black ash moves upward from the bottom inside the furnace, is captured, if a single frame image is used as the object of judgment without considering the passage of time, the upward movement of the ash cannot be grasped, which is recognized as an element indicating the occurrence of caving and may be the main cause of over-detection. According to the present embodiment, regarding the grasp of the state inside the furnace, considering the element of the passage of time, the information of 40 consecutive infrared images acquired by the acquisition unit 40 is input to the long short-term memory network, which is one of the learned models with a recursive structure, for learning, and the state inside the furnace is determined based on the state transition of 40 consecutive infrared images of the infrared image. Thereby, the change in the state inside the furnace that cannot be said to represent caving as described above can be correctly grasped, and thus the occurrence of over-detection can be suppressed.

[0176] (Other embodiments)

[0177] As described above, the embodiments of the present disclosure have been described in detail with reference to the drawings, but the specific structure is not limited to the structures of the respective embodiments, and additions, omissions, replacements, and other changes can be made within the scope not departing from the gist of the present disclosure.

[0178] In addition, the first caving determination unit 75 may also determine whether there is a second-scale caving whose caving scale is larger than that of the first-scale caving based on the two results of the first determination result and the second determination result. In this case, when the sum value of the first caving determination flag value and the second caving determination flag value is equal to or greater than the determination threshold value, the first caving determination unit 75 may determine that there is a second-scale caving.

[0179] In addition, the caving detection unit 41 may also determine whether caving has occurred based on the first infrared image, i.e., the first input element, acquired by the acquisition unit 40 and two or more second infrared images, i.e., the second input elements, captured within the time taken for a single caving of the garbage Fg among a plurality of images in the time series acquired by the acquisition unit 40 before the first infrared image. In this case, the caving detection unit 41 uses the learned model 93 (refer to Figure 2 , hereinafter referred to as the "learned model for caving detection") that has been learned in such a way that when the first input element and the second input element are input, it outputs a determination result related to the possibility of caving occurring for determination. The learned model 93 for caving detection is stored in the storage unit 90 in advance. The caving detection unit 41 obtains the determination result output by inputting the first input element and the second input element received from the acquisition unit 40 to the learned model 93 for caving detection stored in the storage unit 90. The learned model 93 for caving detection is, for example, a deep learning model such as a convolutional neural network. The learned model 93 for caving detection is generated by repeatedly performing a plurality of learning steps, in which the infrared images as the first input element and the second input element captured by the imaging device 2 are input, and the presence or absence of caving corresponding to the infrared image is taught.

[0180] In addition, the images received by the first calculation unit 50 and the second calculation unit 55 from the acquisition unit 40 are not limited to infrared images. In addition, the images received by the third calculation unit 60 and the fourth calculation unit 65 from the acquisition unit 40 are not limited to visible light images.

[0181] In addition, in the embodiment, the incineration device 100 is a grate-type garbage incinerator, but is not limited to a grate-type garbage incinerator. The incineration device 100 may also be, for example, a kiln grate furnace, a biomass fluidized bed boiler, a sludge incinerator, etc. Therefore, the above-described caving detection system 1 may also be a system applied to these incineration devices such as kiln grate furnaces, biomass fluidized bed boilers, and sludge incinerators.

[0182] In addition, Figure 24 is a hardware structure diagram showing the structure of the computer 1100 of the present embodiment. The computer 1100 includes: a processor 1110, a main memory 1120, a memory 1130, and an interface 1140.

[0183] The above-described information processing apparatus 4 is installed in one or more computers 1100. Further, the operations of the above-described respective processing units are stored in the form of programs in the memory 1130. The processor 1110 reads out the programs from the memory 1130 and expands them in the main memory 1120, and executes the above-described processing in accordance with the programs. Further, the processor 1110 ensures a storage area corresponding to the above-described storage unit 90 in the main memory 1120 in accordance with the programs. The programs may also be programs for implementing a part of the functions exhibited by the computer 1100. For example, the programs may function by being combined with other programs already stored in the memory 1130, or by being combined with other programs installed in other devices. Further, the computer 1100 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) on the basis of or instead of the above-described configuration. Examples of the PLD include a PAL (Programmable Array Logic), a GAL (Generic Array Logic), a CPLD (Complex Programmable Logic Device), and an FPGA (Field Programmable Gate Array). In this case, part or all of the functions implemented by the processor 1110 may be implemented by this integrated circuit.

[0184] Examples of the memory 1130 include a magnetic disk, an optical disk, and a semiconductor memory. The memory 1130 may be an internal medium directly connected to the bus of the computer 1100, or may be an external medium connected to the computer 1100 via the interface 1140 or a communication line. Further, when the programs are distributed to the computer 1100 via a communication line, the computer 1100 that has received the distribution may also expand the programs in the main memory 1120 and execute the above-described processing. In the above-described embodiment, the memory 1130 is a non-transitory tangible storage medium. Further, the programs may also be programs for implementing a part of the above-described functions. Also, the programs may also be so-called differential files (differential programs) that implement the above-described functions by being combined with other programs already stored in the memory 1130.

[0185] <Supplementary Note>

[0186] The avalanche detection systems and avalanche detection methods described in each embodiment are understood as follows, for example.

[0187] (1)The collapse detection system 1 of the first mode includes: an acquisition unit 40 that acquires, at a first predetermined cycle, an image obtained by photographing the object to be incinerated (garbage Fg) stacked in the feeder 104 of the incinerator 100 and pushed into the combustion chamber R; a first calculation unit 50 that calculates a representative value of the luminance based on the first image (first infrared image) acquired by the acquisition unit 40; a second calculation unit 55 that calculates a representative value of the luminance based on two or more images (second infrared images) photographed within the time taken for one collapse of the object to be incinerated among a plurality of images in the time series acquired by the acquisition unit 40 before the first image; and a determination unit 70 that makes a determination related to the collapse based on the representative value of the luminance calculated by the first calculation unit 50 and the representative value of the luminance calculated by the second calculation unit 55.

[0188] Thus, compared with the case of determining the presence or absence of collapse based on the luminance of each image acquired at two timings, for example, it is possible to determine the presence or absence of collapse of the garbage Fg with higher accuracy. As a result, it is possible to accurately grasp the combustion state of the garbage Fg in the combustion chamber R.

[0189] (2)On the basis of the collapse detection system 1 of (1), the collapse detection system 1 of the second mode may be such that the second calculation unit 55 calculates the average value or the median value of the luminance related to the two or more images as the representative value of the luminance based on the two or more images.

[0190] Thus, the state of the first infrared image and the state of the second infrared image can be represented by specific statistics.

[0191] (3)On the basis of the collapse detection system 1 of (1) or (2), the collapse detection system 1 of the third mode may be such that the two or more images are images photographed at intervals of a first time, and the two or more images are images photographed before the first image within at least a second time that is twice or more the first time.

[0192] Thus, the above-described operation can be achieved with a more specific setting.

[0193] (4) The collapse detection system 1 of the fourth mode, based on any one of the collapse detection systems 1 in (1) to (3), may also be that the determination unit 70 includes: a first determination unit 71 that makes a first determination related to the collapse based on the representative value of the luminance calculated by the first calculation unit 50 and the representative value of the luminance calculated by the second calculation unit 55; a second determination unit 72 that makes a second determination related to the collapse based on one or more images acquired by the acquisition unit 40; and a first collapse determination unit 75 that determines whether there is a second-scale collapse whose collapse scale is larger than that of the first-scale collapse based on the results of the first determination and the second determination.

[0194] Thereby, the scale of the collapse of the object to be incinerated can be classified. Therefore, the combustion state of the object to be incinerated in the combustion chamber R can be grasped more accurately.

[0195] (5) The collapse detection system 1 of the fifth mode, based on the collapse detection system 1 in (4), may also be that the second determination unit 72 makes the second determination using a learned model (second determination learned model 91) that has been learned in such a way that when an image acquired by the acquisition unit 40 is input, it outputs a determination result related to the possibility of the occurrence of the collapse.

[0196] Thereby, the scale of the collapse of the object to be incinerated can be classified with higher accuracy.

[0197] (6) The collapse detection system 1 of the sixth mode, based on the collapse detection system 1 in (4) or (5), may also be that the acquisition unit 40 acquires an infrared image as the image at the first predetermined period, and acquires a visible light image obtained by photographing the inside of the combustion chamber R at the second predetermined period. The second determination unit 72 makes the second determination based on one or more infrared images acquired by the acquisition unit 40. The determination unit 70 further includes a third determination unit 73 that makes a third determination related to the collapse based on one or more visible light images acquired by the acquisition unit 40. The first collapse determination unit 75 determines whether there is the second-scale collapse based on the results of the first determination, the second determination, and the third determination.

[0198] Thereby, the scale of the collapse of the object to be incinerated can be classified with higher accuracy.

[0199] (7) The collapse detection system 1 of the seventh mode, based on any one of the collapse detection systems 1 in (4) to (6), may also be such that the determination unit 70 includes an interference determination unit 74. The interference determination unit 74 determines the presence or absence of interference based on a feature amount related to the brightness of the image acquired by the acquisition unit 40, and the determination performed by the first collapse determination unit 75 is performed when the interference determination unit 74 determines that there is no interference.

[0200] Thereby, it is possible to suppress a situation where, for example, although the object to be incinerated has not collapsed, a collapse is detected.

[0201] (8) The collapse detection system 1 of the eighth mode, based on any one of the collapse detection systems 1 in (4) to (7), may also be such that the determination unit 70 further includes a second collapse determination unit 76. When the first collapse determination unit 75 determines that there is no collapse of the second scale, the second collapse determination unit 76 determines the presence or absence of a collapse of the first scale based on the change in the brightness of a plurality of images acquired by the acquisition unit 40.

[0202] Thereby, it is possible to classify the scale of the collapse of the object to be incinerated with higher accuracy.

[0203] (9) The collapse detection system 1 of the ninth mode, based on the collapse detection system 1 in (8), may also be such that the second collapse determination unit 76 calculates the similarity between the image acquired by the acquisition unit 40 and one or more images acquired by the acquisition unit 40 in the past, and determines that there is no collapse when the similarity does not satisfy a predetermined condition.

[0204] Thereby, it is possible to further suppress a situation where, for example, although the object to be incinerated has not collapsed, a collapse is still detected, that is, over-detection.

[0205] (10) The collapse detection system 1 of the tenth mode, based on the collapse detection system 1 in (1), may also be such that the determination unit 70 determines the presence or absence of the collapse by performing a process including inputting a feature amount extracted based on one or more images acquired by the acquisition unit 40 into a learned model 97 having a recursive structure.

[0206] Thereby, it is possible to suppress the occurrence of over-detection.

[0207] (11) The collapse detection system 1 of the eleventh mode. On the basis of the collapse detection system 1 in (10), it is also possible that the determination unit 70 includes: a fourth determination unit 77 that performs a fourth determination by inputting a feature amount extracted based on one or more infrared images obtained by the acquisition unit 40 into a deep learning model; a fifth determination unit 78 that performs a fifth determination by inputting a feature amount extracted based on a visible image obtained by the acquisition unit 40 into a deep learning model; and a third collapse determination unit 79 that performs a determination related to the collapse based on a feature amount calculated based on a representative value of brightness calculated by the first calculation unit 50 and a representative value of brightness calculated by the second calculation unit 55, the result of the fourth determination, and the result of the fifth determination.

[0208] (12) The collapse detection system 1 of the twelfth mode. In the collapse detection system 1 in (10) or (11), it is also possible that the determination unit 70 packs the feature amounts of a plurality of images continuously acquired by the acquisition unit 40 at a predetermined time interval, and inputs the packed feature amounts of the plurality of images into the learned model 97 having the recursive structure.

[0209] (13) The collapse detection system 1 of the thirteenth mode. On the basis of the collapse detection system 1 in (12), it is also possible that an autoencoder 96 is used to reduce the dimensions of a plurality of images continuously acquired by the acquisition unit 40 at a predetermined time interval, and the feature amounts of the plurality of images with reduced dimensions are packed.

[0210] (14) The collapse detection method of the fourteenth mode includes the following steps: one or more computers 1100 acquire, at a first predetermined period, images obtained by photographing the incinerated material stacked in the feeder 104 of the incineration device 100 and pushed into the combustion chamber R; calculate a representative value of the brightness based on the acquired first image; calculate representative values of the brightness based on two or more images captured within the time taken for one collapse of the incinerated material among a plurality of images in the time series acquired before the first image; and perform a determination related to the collapse based on the representative value of the brightness based on the first image and the representative values of the brightness based on the two or more images.

[0211] (15) The collapse detection system 1 of the fifteenth mode includes: an acquisition unit 40 that acquires, at a first predetermined period, images obtained by photographing the incinerated material stacked in the feeder 104 of the incineration device 100 and pushed into the combustion chamber R; and a collapse detection unit 41 that performs a determination related to the collapse based on the first image, i.e., the first input element, obtained by the acquisition unit 40 and two or more images, i.e., the second input element, captured within the time taken for one collapse of the incinerated material among a plurality of images in the time series acquired by the acquisition unit 40 before the first image.

[0212] [Industrial Applicability]

[0213] The present disclosure relates to a system for detecting the collapse of an object to be incinerated in a combustion chamber of an incinerator. According to the collapse detection system of the present disclosure, the accuracy of detecting the collapse of the object to be incinerated can be improved.

[0214] Description of Reference Numerals

[0215] 1... Collapse detection system

[0216] 2... Imaging device

[0217] 4... Information processing device

[0218] 5... Infrared camera

[0219] 6... Visible light camera

[0220] 8... Filtering device

[0221] 40... Acquisition unit

[0222] 41... Collapse detection unit

[0223] 42... First object area

[0224] 43... Second object area

[0225] 44... Third object area

[0226] 45, 45a, 45b, 45c... Fourth object area

[0227] 46... Fifth object area

[0228] 46′... Binarized data

[0229] 42a... First area

[0230] 42b... Second area

[0231] 42c... Third area

[0232] 42d... Fourth area

[0233] 42e... Fifth area

[0234] 42f... Sixth area

[0235] 42g... Seventh area

[0236] 42h... Eighth area

[0237] 42i... Ninth area

[0238] 43c... Central area

[0239] 43l…Left region

[0240] 43r…Right region

[0241] 50…First calculation unit

[0242] 55…Second calculation unit

[0243] 60…Third calculation unit

[0244] 65…Fourth calculation unit

[0245] 66…Fifth calculation unit

[0246] 67…Sixth calculation unit

[0247] 70…Judgment unit

[0248] 71…First judgment unit

[0249] 72…Second judgment unit

[0250] 73…Third judgment unit

[0251] 74…Interference judgment unit

[0252] 75…First collapse judgment unit

[0253] 76…Second collapse judgment unit

[0254] 77…Fourth judgment unit

[0255] 78…Fifth judgment unit

[0256] 79…Third collapse judgment unit

[0257] 80…Control unit

[0258] 81…Fourth collapse judgment unit

[0259] 90…Storage unit

[0260] 91…Learned model for second judgment

[0261] 92…Learned model for interference judgment

[0262] 93…Learned model for collapse detection

[0263] 94…Learned model for fourth judgment

[0264] 95…Learned model for fifth judgment

[0265] 96…Autoencoder

[0266] 97…Learned model for collapse judgment

[0267] 100…Incineration equipment

[0268] 102… Hopper

[0269] 104… Feeder

[0270] 108… Furnace main body

[0271] 110… Pushing device

[0272] 112… Air supply device

[0273] 114… Heat recovery boiler

[0274] 116… Cooling tower

[0275] 118… Dust collection device

[0276] 120… Chimney

[0277] 121… Downstream end

[0278] 122… Receiving port

[0279] 124… Pushing arm

[0280] 126… Grate bar

[0281] 128… Drying area

[0282] 130… Combustion area

[0283] 131… Flame

[0284] 132… Afterburning area

[0285] 135… Ash

[0286] 136… Air supply pipe

[0287] 138… Blower

[0288] 140… First flow control valve

[0289] 142… Second flow control valve

[0290] 143… Exhaust gas

[0291] 144… Flue

[0292] 145… Furnace tail

[0293] 146… Ash chute

[0294] 1100… Computer

[0295] 1110… Processor

[0296] 1120… Main memory

[0297] 1130… Memory

[0298] 1140… Interface

[0299] Fg… Garbage (Incinerated Material)

[0300] Fr… Front Surface

[0301] R… Combustion Chamber

[0302] S… Controlled Object Device

[0303] W1… Conveying Direction.

Claims

1. A caving detection system, comprising: An acquisition unit that acquires, at a first predetermined period, an image obtained by photographing combustibles stacked in a feeder of an incinerator and pushed into a combustion chamber; A first calculation unit that calculates a representative value of the brightness based on the first image acquired by the acquisition unit; A second calculation unit that calculates a representative value of the brightness based on two or more images among a plurality of images in a time series acquired by the acquisition unit before the first image and photographed within the time taken for one caving of the combustibles; And A determination unit that makes a determination related to the caving based on the representative value of the brightness calculated by the first calculation unit and the representative value of the brightness calculated by the second calculation unit.

2. The caving detection system according to claim 1, Wherein, The second calculation unit calculates an average value or a median value of the brightness related to the two or more images as the representative value of the brightness based on the two or more images.

3. The caving detection system according to claim 1 or 2, Wherein, The two or more images are images photographed at intervals of a first time, The two or more images are images photographed before the first image within a second time that is at least twice the first time.

4. The caving detection system according to claim 1 or 2, Wherein, The determination unit includes: A first determination unit that makes a first determination related to the caving based on the representative value of the brightness calculated by the first calculation unit and the representative value of the brightness calculated by the second calculation unit; A second determination unit that makes a second determination related to the caving based on one or more images acquired by the acquisition unit; And A first caving determination unit that determines whether there is a second-scale caving whose caving scale is larger than that of a first-scale caving based on the result of the first determination and the result of the second determination.

5. The caving detection system according to claim 4, Wherein, The second determination unit uses a learned model that has been learned in such a way that when an image acquired by the acquisition unit is input, it outputs a determination result related to the possibility of the occurrence of the caving to make the second determination.

6. The caving detection system according to claim 4, Wherein, The acquisition unit acquires an infrared image as the image at the first predetermined period and acquires a visible light image obtained by photographing the combustion chamber at a second predetermined period, The second determination unit makes the second determination based on one or more infrared images acquired by the acquisition unit, The determination unit further includes a third determination unit that makes a third determination related to the caving based on one or more visible light images acquired by the acquisition unit, The first caving determination unit determines whether there is the second-scale caving based on the result of the first determination, the result of the second determination, and the result of the third determination.

7. The caving detection system according to claim 4, Wherein, The determination unit includes an interference determination unit that determines whether there is interference based on a feature amount related to the brightness of the image acquired by the acquisition unit. The determination performed by the first caving determination unit is carried out when the interference determination unit determines that there is no interference.

8. The caving detection system according to claim 4, wherein, the determination unit further includes a second caving determination unit. When the first caving determination unit determines that there is no caving of the second scale, the second caving determination unit determines whether there is caving of the first scale based on the change in the brightness of a plurality of images acquired by the acquisition unit.

9. The caving detection system according to claim 8, wherein, the second caving determination unit calculates the similarity between the image acquired by the acquisition unit and one or more images acquired by the acquisition unit in the past. When the similarity does not meet a predetermined condition, it is determined that there is no caving.

10. The caving detection system according to claim 1, wherein, the determination unit determines whether there is the caving by performing a process including inputting a feature amount extracted based on one or more images acquired by the acquisition unit into a learned model having a recursive structure.

11. The caving detection system according to claim 10, wherein, the determination unit includes: a fourth determination unit that performs a fourth determination by inputting a feature amount extracted based on one or more infrared images acquired by the acquisition unit into a deep learning model; a fifth determination unit that performs a fifth determination by inputting a feature amount extracted based on visible images acquired by the acquisition unit into a deep learning model; and a third caving determination unit that performs a determination related to the caving based on a feature amount calculated based on a representative value of brightness calculated by the first calculation unit and a representative value of brightness calculated by the second calculation unit, the result of the fourth determination, and the result of the fifth determination.

12. The caving detection system according to claim 10 or 11, wherein, the determination unit packs the feature amounts of a plurality of images continuously acquired by the acquisition unit at a predetermined time interval, and inputs the packed feature amounts of the plurality of images into the learned model having the recursive structure.

13. The caving detection system according to claim 12, wherein, an autoencoder is used to reduce the dimensions of a plurality of images continuously acquired by the acquisition unit at a predetermined time interval, and the feature amounts of the plurality of images with reduced dimensions are packed.

14. A caving detection method, including the following steps: One or more computers acquire images obtained by photographing the material to be incinerated stacked in the feeder of the incineration device and pushed into the combustion chamber at a first predetermined period; Calculate a representative value of the brightness based on the acquired first image; Calculate a representative value of the brightness based on two or more images taken within the time taken for a single caving of the material to be incinerated among a plurality of images in the time series acquired before the first image; Perform a determination related to the caving based on the representative value of the brightness based on the first image and the representative value of the brightness based on the two or more images.

15. A caving detection system, comprising: An acquisition unit that acquires, at a first predetermined period, an image obtained by photographing the object to be incinerated that is stacked in a feeder of an incinerator and pushed into a combustion chamber; and A collapse detection unit that makes a determination related to the collapse based on a first image acquired by the acquisition unit, which is a first input element, and two or more images acquired by the acquisition unit in a time series before the first image and photographed within the time taken for a single collapse of the object to be incinerated, which are second input elements.

Images