Information processing apparatus, information processing method, and program

Abstract

To analyze which information of information included in a picked-up image affects a situation of an object.SOLUTION: A control part acquires environmental data about an environment for predicting an occurrence of a change in an object or prescribed change caused by the change, acquires environmental data related to the prescribed change after a prescribed lapse time from a prescribed time point within a range that goes back to before a first retroactive time, acquires a plurality of pieces of picked-up image data picked up by an imaging part with a range that goes back to before a second retroactive time, extracts a plurality of feature amounts digitized from the respective pieces of the picked-up image data, generates an image generation learning model on the basis the extracted feature amounts and the acquired environmental data, selects at least one feature amount having a large correlation to the change in the object or the prescribed change caused by the change among the plurality of feature amounts, inputs the extracted feature amounts to the image generation learning model to generate feature image data, and changes a numerical value of the selected feature amount to generate feature change image data obtained by changing the feature image data.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, various techniques have been proposed for controlling various processes in the operation of a plant by using information included in a captured image obtained by capturing a part of the plant.

[0003] Patent Document 1 proposes a technique for displaying adjacent to each other a measured luminance or temperature distribution and a result obtained by image processing in a combustion diagnosis apparatus.

[0004] Patent Document 2 discloses a configuration in which image data of a flame in a normal combustion state or a feature amount of the image data of a normal flame is acquired, and while comparing with the image data of the flame of a current combustion device or the feature amount of the current flame image data, it is displayed, and the similarity between the feature amount of the image data of the flame in the acquired normal combustion state and the feature amount of the image data of the flame of the current combustion device is evaluated to detect whether the combustion state of the current combustion device is normal.

[0005] Patent Document 3 proposes a technique for creating a model showing the relationship between input parameters and process values of a plant that burns fuel by reading operation data of the plant including at least one of physical parameters related to the plant specifications or fuel parameters related to the properties of the fuel, and image data of the combustion region of the plant, extracting feature amounts of the image data, and using at least one of the physical parameters or fuel parameters and the feature amounts of the image data as input parameters.

[0006] Patent Document 4 proposes an information processing apparatus including a flame region extraction unit that extracts a flame region in which an image of a flame appears from each of a plurality of captured images obtained by capturing the inside of a garbage incinerator in time series, and a base portion specifying unit that specifies, as a base portion of the flame, a portion of the flame region extracted by the flame region extraction unit that has relatively little temporal change.

Prior Art Documents

Patent Documents

[0007]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0008] A control technique for adjusting various controls for the operation of a target facility or object (hereinafter collectively referred to as the object), such as a plant, based on information included in an image, as proposed by the above-described prior art, is referred to as an image control technique in this specification. However, regarding this image control technique, in the above-described prior art, a method for clarifying which information included in the acquired image affects the situation of the object has not been studied. As a result, in the prior art, the situations where processing using an image can be applied have been limited. Therefore, a technique capable of analyzing which information among the information included in the captured image affects the situation of the object has been demanded.

[0009] The present invention has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and a program capable of analyzing which information among the information included in a captured image affects the situation of an object.

Means for Solving the Problems

[0010] In order to solve the above-described problems and achieve the object, an information processing apparatus according to an aspect of the present invention includes a control unit having hardware. The control unit acquires environmental data related to an environment in which the object exists for predicting the occurrence of a change in the object or a predetermined change caused by the change in the object, and stores the environmental data in a storage unit. The environmental data related to the change in the object or the predetermined change caused by the change in the object after a predetermined elapsed time from a predetermined time point is read from the storage unit within a range retroactively from the predetermined time point to a predetermined first retroactive time. Imaging image data generated by imaging the object by an imaging unit is acquired and stored in the storage unit in association with information at the imaging time. A plurality of the imaging image data is read from the storage unit within a range retroactively from the predetermined time point to a predetermined second retroactive time. A plurality of quantified feature amounts are extracted from each of the read plurality of imaging image data and stored in the storage unit. At least one feature amount having a large correlation with the change in the object or a predetermined change caused by the change in the object among the plurality of feature amounts is selected. The extracted feature amounts are input to an image generation learning model that generates image data to generate feature image data. The feature image data is changed by changing the numerical values of the selected feature amounts to generate and output feature change image data.

[0011] In the information processing apparatus according to an aspect of the present invention, in the above invention, the control unit inputs, as input parameters, a feature amount obtained by averaging the plurality of quantified feature amounts to the image generation learning model, and causes the image generation learning model to output, as output parameters, averaged feature image data.

[0012] In the information processing apparatus according to an aspect of the present invention, in the above invention, the control unit derives a difference between the averaged feature image data and the feature change image data, and extracts a factor of the predetermined change in the feature change image data.

[0013] In the information processing apparatus according to an aspect of the present invention, in the above invention, the difference is a difference in luminance.

[0014] An information processing apparatus according to one aspect of the present invention is, in the above invention, wherein the object is waste in an incinerator, and the environmental data in the environment where the object exists is process data in the incinerator.

[0015] An information processing apparatus according to one aspect of the present invention is, in the above invention, wherein a predetermined change caused by a change in the object is a change in the temperature inside the incinerator or a change in the concentration of nitrogen oxide or carbon monoxide discharged from the incinerator.

[0016] An information processing apparatus according to one aspect of the present invention is, in the above invention, wherein the first retrospective time and the second retrospective time are the same time.

[0017] An information processing method according to one aspect of the present invention is an information processing method executed by an information processing apparatus including a control unit having hardware, the method including: acquiring environmental data related to an environment where an object exists for predicting the occurrence of a change in the object or a predetermined change caused by the change in the object, and storing the environmental data in a storage unit; reading, from the storage unit, the environmental data related to the predetermined change that occurs in the object after a predetermined elapsed time from a predetermined time, within a range retroactively from the predetermined time to a predetermined first retrospective time; acquiring imaging image data generated by imaging the object by an imaging unit, and storing the imaging image data in the storage unit in association with information at the imaging time; reading, from the storage unit, a plurality of the imaging image data within a range retroactively from the predetermined time to a predetermined second retrospective time; extracting a plurality of quantified feature amounts from each of the read plurality of imaging image data and storing the feature amounts in the storage unit; selecting at least one feature amount having a large correlation with the change in the object or a predetermined change caused by the change in the object among the plurality of feature amounts; inputting the extracted feature amounts into an image generation learning model for generating image data to generate feature image data; and changing numerical values of the selected feature amounts to generate feature change image data obtained by changing the feature image data.

[0018] A program according to an aspect of the present invention causes a control unit in an information processing apparatus having hardware to acquire environmental data related to an environment in which an object exists for predicting the occurrence of a change in the object or a predetermined change resulting from the change in the object, and stores the acquired environmental data in a storage unit. The environmental data related to the predetermined change occurring in the object after a predetermined elapsed time from a predetermined time is read from the storage unit within a range retroactively from the predetermined time to a predetermined first retroactive time. Image capture image data generated by capturing the object by an image capture unit is acquired and stored in the storage unit in association with information on the image capture time. A plurality of the image capture image data is read from the storage unit within a range retroactively from the predetermined time to a predetermined second retroactive time. A plurality of quantified feature amounts are extracted from each of the read plurality of image capture image data and stored in the storage unit. At least one feature amount having a large correlation with the change in the object or a predetermined change resulting from the change in the object among the plurality of feature amounts is selected. The extracted feature amounts are input to an image generation learning model for generating image data to generate feature image data. The numerical values of the selected feature amounts are changed to generate feature change image data obtained by changing the feature image data.

Advantages of the Invention

[0019] According to the information processing apparatus, information processing method, and program according to the present invention, it is possible to analyze which information among the information included in the captured image affects the situation of the object.

Brief Description of the Drawings

[0020]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

[0021] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings of the following embodiment, the same or corresponding parts are denoted by the same reference numerals. Also, the present invention is not limited to the embodiment described below.

[0022]

[0023] First, in explaining an image processing device according to an embodiment of the present invention, the intensive studies conducted by the present inventor will be described. That is, according to the findings of the present inventor, in the prior art, while techniques for adding various devices to the control of objects such as plants have been proposed, no method has been studied to clearly identify, through learning, which information among the information included in the acquired image affects the situation of the object.Specifically, for example, a technology for adjusting the control of an object using information included in a captured image of the object such as a plant (hereinafter referred to as an image control technology) is applicable only in the following situations. That is, first, the image control technology is applicable when it is known what kind of influence the information included in the captured image has on the situation of the object. Second, the image control technology is applicable when the causal relationship between the information included in the captured image and the situation of the plant as the object is unclear, but there is a learning model or model that associates both the information included in the captured image and the situation of the object by means of machine learning or the like. However, it has not been considered to clarify by means of machine learning or the like which of the information included in the captured image can affect the situation of the object.

[0024] Therefore, the present inventor has intensively studied a method for clarifying the cause of fluctuations in the situation of a plant, such as a sudden drop in the temperature of a certain part, regarding the situation of the plant as the object. Then, the present inventor came up with the idea of modeling the relationship between the feature amount composed of a plurality of vector components extracted from the captured image and the fluctuations of the object such as a plant. Furthermore, in the model, the present inventor derived a feature amount that emphasizes the vector having a strong relationship between the plurality of feature amounts and the fluctuations of the object, and generated and drew a captured image based on the derived feature amount, thereby devising an image processing method for explicitly showing the factor of fluctuations in the object by an image.

[0025] The present inventor performed the following processing to realize the devised image processing method. That is, first, a feature amount space is generated using, for example, a plurality of captured images of about 10,000 data. Next, the generated feature amount space and the process data of the plant, which is an example of the environmental data related to the environment where the object exists, are input into a prediction model using a machine learning framework (for example, Light GBM: Light Gradient Boosting Machine) that handles a gradient boosting algorithm based on a decision tree algorithm.

[0026] Next, to explain the individual prediction results of the model, a contribution degree of each feature quantity to the prediction is calculated using a processing method for explaining, for example, SHAP (Shapley Additive exPlanations). By the sum of the contributions of each feature quantity being consistent with the predicted value, the contribution degree of each feature quantity to the prediction can be calculated. As an output example of SHAP, by calculating the SHAP value for each piece of all data and visualizing the overall image, an output that allows an overview of the whole can be obtained. Also, by visualizing the relationship between the value of a feature quantity and the SHAP value for one feature quantity, it becomes possible to visualize the contribution to the prediction for a specific feature quantity. Further, by visualizing how each feature quantity acts on one piece of data, it becomes possible to visualize the contribution of each feature quantity to one piece of data.

[0027] After that, in the feature quantity space generated by these methods, a feature quantity with a strong correlation to a predetermined change is changed, and for example, by unsupervised learning such as a generative adversarial network typified by GAN (Generative Adversarial Network), the relationship between the feature quantity and the image is visualized and output. Thereby, an image corresponding to the variation of the object can be generated and output. Note that a generative adversarial network such as GAN can generate non-existent data or transform along the features of existing data by learning features from the data. Also, a prediction model predicts and outputs future variations in the object. Note that instead of GAN, it is also possible to use DCGAN (Deep Convolutional GAN) that uses a convolutional neural network (CNN: Convolutional Neural Network) for unsupervised learning, VAE (Variational Autoencoder), or the like.

[0028] Specifically, for example, in order to extract the variation pattern over time of the feature quantity corresponding to the temperature variation in the process data of a plant, the following processing is executed. That is, first, a plurality of image data related to combustion in the plant, for example, about 10,000 time-series combustion images traced back in the past for a predetermined time of about 5 minutes from the present, are acquired. Next, as the first processing, learning is performed from the image data by unsupervised learning such as GAN. Subsequently, second, learning is performed on a model (for example, CNN) that maps an image to a feature quantity space configured in GAN or the like. Third, learning is performed by a prediction model (for example, LightGBM) using the data obtained by converting the image data into feature quantities and the process data. In other words, a correlation with the process data accompanying the temperature variation is learned to construct a prediction model for predicting a sudden temperature drop. Next, as the fourth, from the constructed prediction model such as LightGBM, the relationship between the feature quantity and the sudden temperature drop is extracted using an explainable AI method such as SHAP. For example, the feature quantity contributing to the prediction of the sudden temperature drop is extracted to extract the relationship between the feature quantity and the temperature. Fifth, for example, the relationship between the feature quantity and the image is visualized and output by GAN. Note that adversarial generative networks such as GAN can generate non-existent data or transform along the features of existing data by learning features from the data.

[0029] That is, the inventor has devised an image processing method that extracts feature quantities from a plurality of acquired actual captured images and generates an image using the extracted feature quantities in order to explainably show the location of the cause of the change in the state of the object by an image with respect to the change in the state of the object. Then, by applying this image processing method and associating the extracted feature quantities with the process data related to the object, an image prediction method is devised that predicts the variation of the situation of the object after a predetermined time and predicts it by an image. As a result, since the variation of the situation of the object can be predicted, the inventor has conceived a method that can efficiently control the object by using the extracted feature quantities and the images generated from the feature quantities for the control related to the object.

[0030] One embodiment described below has been devised based on the inventor's intensive studies as described above, and will be described by taking as an example the case where the object is waste to be incinerated in a grate incinerator.

[0031] (Grate incinerator) FIG. 1 shows a grate incinerator to which an information processing apparatus according to an embodiment of the present invention is applied. As shown in FIG. 1, the grate incinerator as a waste incinerator includes an incinerator 1 where waste is burned, a waste inlet 2 for introducing waste, and a boiler 9. The boiler 9 as a steam generation unit includes a heat exchanger 9a and a steam drum 9b installed downstream of the furnace outlet 7 of the incinerator 1.

[0032] The waste introduced from the waste inlet 2 is conveyed to the grate 4 by a waste supply device 3 as a waste supply means. By the reciprocating motion of the grate 4, the waste is agitated and moved. The waste on the grate 4 is burned while being dried by the blowing of combustion air supplied by a combustion air blower 6 into the air box below the grate 4, and exhaust gas and ash are generated. The generated ash falls through the ash fall port 5 and is discharged to the outside of the incinerator 1.

[0033] The total amount of combustion air supplied into the incinerator 1 from below the grate 4 is adjusted by a combustion air damper 14 provided in the immediate vicinity of the combustion air blower 6 as a forced air blower. The flow rate of the combustion air supplied to each air box is adjusted by combustion air dampers 14a, 14b, 14c, 14d provided respectively in the pipes supplying combustion air to the respective air boxes. In other words, the ratio of the flow rate of the combustion air supplied to each air box is adjusted by the combustion air dampers 14a to 14d under the grate. In FIG. 1, the area below the grate 4 is divided into four air boxes along the waste conveyance direction, and combustion air is supplied through each air box, but the number of the combustion air dampers 14a to 14d under the grate and the number of air boxes are not necessarily limited to four, and can be appropriately changed according to the scale and purpose of the grate incinerator.

[0034] Secondary air is blown into the incinerator 1 from a secondary air inlet 10 provided in the furnace wall 1a (see Fig. 2) by a secondary air blower 11 serving as a secondary air blower. When the secondary air is blown into the incinerator 1, unburned components in the combustion gas are further burned, and an excessive rise in the temperature of the furnace wall 1a is suppressed. The flow rate of the secondary air supplied from the secondary air inlet 10 into the incinerator 1 is adjusted by a secondary air damper 15 provided in the immediate vicinity of the secondary air blower 11.

[0035] Along the conveyance direction of the waste on the grate 4, the combustible gas generated in the upstream waste drying process (drying stage) and main combustion process (combustion stage), and the combustion exhaust gas generated in the downstream afterburning process (afterburning stage) merge at a gas mixing section provided on the furnace outlet 7 side of the incinerator 1. The combustible gas and combustion exhaust gas merged at the gas mixing section are stirred and mixed again, and then secondary combustion is carried out by supplying secondary combustion air. The boiler 9 is installed downstream along the conveyance direction of the waste with respect to the portion where secondary combustion is carried out (hereinafter referred to as the secondary combustion section). The combustion gas after secondary combustion has its thermal energy recovered by the heat exchanger 9a of the boiler 9 and is then exhausted to the outside from the chimney 8.

[0036] An intermediate ceiling 16 is provided at an upper position along the height direction of the incinerator 1 inside the incinerator 1. The gas flowing inside the incinerator 1 can be divided and discharged by the intermediate ceiling 16 into a gas containing a large amount of combustible gas generated in the waste drying process and main combustion process on the upstream side and a combustion exhaust gas generated in the afterburning process on the downstream side. Specifically, while the combustion exhaust gas flows through a flue (main flue) below the intermediate ceiling 16, the gas containing a large amount of combustible gas flows through a flue (sub flue) above the intermediate ceiling 16. By the combustion exhaust gas and the gas containing a large amount of combustible gas merging at the gas mixing section, the stirring and mixing of the gas at the gas mixing section are further promoted. Thereby, the combustion in the secondary combustion section becomes more stable, the generation of dioxins in the combustion process can be suppressed, and the generation of unburned components of the waste can be suppressed. Note that the incinerator 1 may be configured without the intermediate ceiling 16 provided therein.

[0037] Thermometers are provided at multiple positions within the incinerator 1 as sensors for measuring the gas temperature within the incinerator 1. Specifically, along the height direction of the incinerator 1, a combustion chamber gas thermometer 17 is provided at an intermediate position between the fire grate 4 and the secondary air inlet 10. Along the height direction of the incinerator 1, a main flue gas thermometer 18 is provided at a position below the furnace outlet 7. Along the height direction of the incinerator 1, a lower furnace outlet gas thermometer 19 is provided at the lower position of the furnace outlet 7. Along the height direction of the incinerator 1, a middle furnace outlet gas thermometer 20 is provided at the middle position of the furnace outlet 7. Along the height direction of the incinerator 1, a furnace outlet gas thermometer 21 for measuring the combustion management temperature is provided at the downstream side position of the furnace outlet 7. The measured values of the temperatures measured by the combustion chamber gas thermometer 17, the main flue gas thermometer 18, the lower furnace outlet gas thermometer 19, the middle furnace outlet gas thermometer 20, and the furnace outlet gas thermometer 21 are transmitted to the combustion control device 30 as combustion process measurement values and stored in the storage unit 32 (see FIG. 4).

[0038] The boiler 9 is provided with a boiler outlet oxygen concentration meter 22 for measuring the concentration of oxygen (O2) in the exhaust gas on the outlet side. At the inlet of the chimney 8, a gas concentration meter 23 for measuring the concentrations of carbon monoxide (CO) and nitrogen oxides (NO x ) is provided. A flue gas flow meter 24 for measuring the flue gas volume is provided in the pipe connecting the outlet of the boiler 9 and the chimney 8. The measured values of the concentrations and flow rates of the gas measured by the boiler outlet oxygen concentration meter 22, the gas concentration meter 23, and the flue gas flow meter 24 are transmitted to the combustion control device 30 as combustion process measurement values and stored in the storage unit 32. Further, the boiler 9 is provided with a steam flow meter 25 for measuring the amount of steam generated in the boiler 9. The measured value of the steam generation amount of the boiler 9 measured by the steam flow meter 25 is transmitted to the combustion control device 30 as a combustion process measurement value and stored in the storage unit 32.

[0039] On the downstream side in the waste conveyance direction in the incinerator 1, an imaging unit 26 is provided. The imaging unit 26 is configured to include, for example, a flame transmission camera composed of an infrared camera and an image processing unit that processes the captured image data. FIG. 2 is a side view showing the installation state of the imaging unit 26. The imaging unit 26 may be disposed outside the furnace in proximity to a monitoring window provided in the furnace wall 1a, or may be disposed inside the incinerator 1 having a water-cooled structure. As shown in FIG. 2, the waste 50 drops from the waste supply unit 12 onto the fire grate 4 at the step wall 13. The waste 50 that has dropped onto the fire grate 4 is moved forward on the side of the imaging unit 26 while being agitated by the reciprocating motion accompanying the forward and backward movement of the fire grate 4.

[0040] The imaging unit 26 can acquire the thermographic information of the waste 50 on the fire grate 4 (hereinafter, the waste on the fire grate 52) as thermal image information. Here, the wavelength of the infrared rays radiated from the waste 50 is different from the wavelength of the infrared rays radiated from the high-temperature gas and the flame in the space. Therefore, in the imaging unit 26, even if there is a flame in the measurement field of view by appropriately selecting the infrared wavelength to be measured, it is possible to obtain thermal image information corresponding to the temperature distribution of the layer of the waste 52 on the fire grate. Further, by setting the measurement range in the furnace length direction by the imaging unit 26, it is possible to obtain the thermal image information of the layer of the waste 52 on the fire grate at a position upstream of the combustion region (upstream of the flame). The thermal image information can be treated as video data in a state where the flame has passed through, that is, a plurality of image data.

[0041] In other words, the imaging unit 26 can image the waste 50 sent out from the waste supply unit 12 (hereinafter referred to as the pre-supply waste 51), the step wall 13 having a step where the waste 50 drops, the waste 52 on the grate, and the upper surface of the grate 4 in a state where the flame has passed through. Note that a combustion image imaging unit for imaging the combustion state of the waste 52 on the grate, that is, the flame itself, may be further provided. The imaging image data (hereinafter referred to as transmitted image data) imaged in a state where the flame imaged by the imaging unit 26 has passed through is transmitted to the image processing device 40 immediately or at a predetermined time interval. Note that the transmitted image data imaged by the imaging unit 26 may be stored in the storage unit 32 of the combustion control device 30 and then transmitted from the combustion control device 30 to the image processing device 40.

[0042] In the present embodiment, the imaging unit 26 is installed, for example, at a position substantially directly facing the waste supply unit 12 and the step wall 13. Note that the installation of the imaging unit 26 is not limited to a position substantially directly facing the waste supply unit 12 and the step wall 13. The installation position of the imaging unit 26 can be set at various positions as long as at least the boundary portion between the waste 52 on the grate and other objects, here the step wall 13 and the grate 4, can be imaged.

[0043] FIG. 3 is a front view showing an example of the visual field of the imaging unit 26. As shown in FIG. 3, the imaging unit 26 has, for example, a measurement visual field that extends in the vertical direction and the furnace width direction (left-right direction) of the incinerator 1. In the present embodiment, the visual field of the imaging unit 26 includes the waste supply unit 12, the step wall 13, the grate 4, and the furnace wall 1a. The furnace wall 1a included in the visual field of the imaging unit 26 restricts the movement, that is, the spread of the waste 50 to the outside in the left-right direction. Note that the visual field of the imaging unit 26 only needs to have a visual field capable of imaging the boundary portion between the waste 52 on the grate existing on the grate 4 and the grate 4 and the step wall 13. Further, it is preferable that the imaging unit 26 can image the waste 50 conveyed to the waste supply unit 12. Thereby, the waste 50 dropping at the position of the step wall 13 can be imaged.

[0044] FIG. 4 is a block diagram showing the configurations of the combustion control device 30 and the image processing device 40. The combustion control device 30 and the image processing device 40 are connected via a network (not shown) composed of, for example, a dedicated line, a public communication network such as the Internet, for example, a combination of one or more of a telephone communication network such as a LAN (Local Area Network), a WAN (Wide Area Network), and a mobile phone, a public line, and a VPN (Virtual Private Network). Further, the combustion control device 30 and the image processing device 40 may be integrally configured, and the combustion control device 30 and the image processing device 40 may be installed in the same facility as the grate incinerator or in separate facilities. When the grate incinerator, the combustion control device 30, and the image processing device 40 are installed in separate facilities, communication of various information and various data is performed via the above-described network.

[0045] As shown in FIG. 4, the combustion control device 30 includes a control unit 31, a storage unit 32, and an operation amount adjustment unit 33. Specifically, the control unit 31 as a combustion control unit and the operation amount adjustment unit 33 include a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field-Programmable Gate Array) having hardware, and a main storage unit (both not shown) such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The storage unit 32 is composed of a storage medium selected from a volatile memory such as a RAM, a non-volatile memory such as a ROM, an EPROM (Erasable Programmable ROM), a hard disk drive (HDD), and a removable medium. The removable medium is, for example, a USB (Universal Serial Bus) memory or a disk recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a BD (Blu-ray (registered trademark) Disc). Alternatively, the storage unit 32 may be configured using a computer-readable recording medium such as a memory card that can be externally attached.

[0046] The storage unit 32 can store an operating system (OS), various programs, various tables, various databases, etc., for executing the operations of the combustion control device 30. Here, the various programs also include an information processing program that realizes processing based on models such as the learning model and the learned model according to the present embodiment. These various programs can also be recorded on a computer-readable recording medium such as a hard disk, a flash memory, a CD-ROM, a DVD-ROM, or a flexible disk and widely distributed.

[0047] The combustion control device 30 controls, based on a predetermined operating quantity reference value setting relational expression (hereinafter referred to as the operating quantity relational expression), the waste supply device feed rate (also referred to as the dust supply rate) for adjusting the waste supply rate of the waste 50 and the grate feed rate (also referred to as the grate speed) for adjusting the moving speed of the waste 50 as the operating quantities of the respective operating ends. Note that the combustion control device 30 also controls the stop and operation of the waste supply device feed rate and the grate feed rate. The combustion control device 30 controls the combustion air quantity and the secondary air quantity based on the operating quantity relational expression as needed. The operating quantity relational expression is, for example, a relational expression between a waste incineration quantity set value or a waste substance set value and an operating quantity reference value (target value of the operating quantity), and includes control parameters as correction coefficients. The control parameters are adjusted by the control unit 31 to conform to the waste incineration quantity set value and the waste substance set value. When at least one of the waste incineration quantity set value and the waste substance set value is changed, the adjusted control parameters are changed by the control unit 31 corresponding to the changed set value. When the control parameters are changed, the preset operating quantity reference value is corrected.

[0048] The control unit 31 calculates the waste substance (lower calorific value of the waste) according to the waste incineration quantity set value. The control unit 31 adjusts the operating quantity reference value by adjusting the control parameters included in the operating quantity relational expression. The control unit 31 corrects the adjusted operating quantity reference value based on a predetermined control algorithm such as PID control or fuzzy calculation. The storage unit 32 stores the data referred to by the control unit 31. The storage unit 32 stores a predetermined operating quantity relational expression, a control algorithm, a preset incineration quantity set value, and a combustion process measurement value obtained as a combustion state quantity in the incinerator 1.

[0049] The operating quantity adjustment unit 33 adjusts the respective operating quantities of the respective operating ends so as to follow the operating quantity reference value. Specifically, the operating quantity adjustment unit 33 includes a combustion air quantity adjustment unit 331, an air quantity ratio adjustment unit 332, a secondary air quantity adjustment unit 333, a waste supply device feed rate adjustment unit 334, and a grate feed rate adjustment unit 335.

[0050] The combustion air quantity adjustment unit 331 adjusts the operation quantity so as to follow the operation quantity reference value (hereinafter referred to as the corrected operation quantity reference value) in which the combustion air quantity is corrected by the control unit 31. The air quantity ratio adjustment unit 332 controls each of the under-grate combustion air dampers 14a to 14d to adjust the mutual ratio of the flow rates in the respective wind boxes. The secondary air quantity adjustment unit 333 adjusts the operation quantity so that the secondary air quantity follows the corrected operation quantity reference value. Here, the adjustment of the combustion air quantity and the secondary air quantity is performed by controlling the opening degrees of the combustion air damper 14, the under-grate combustion air dampers 14a to 14d, and the secondary air damper 15, respectively.

[0051] The waste supply device feed speed adjustment unit 334 adjusts the operation quantity so that the waste supply device feed speed follows the corrected operation quantity reference value. The grate feed speed adjustment unit 335 adjusts the operation quantity so that the grate feed speed follows the corrected operation quantity reference value. When the operation quantity reference value is not corrected by the control unit 31, the operation quantity adjustment unit 33 adjusts each operation quantity based on the uncorrected operation quantity reference value.

[0052] (Image processing device) The image processing device 40 as an information processing device includes a control unit 41, an output unit 42, an input unit 43, and a storage unit 44. The image processing device 40 functions as a feature quantity extraction device that acquires a plurality of transmission image data and extracts a feature quantity from the acquired plurality of combustion image data, and selects a feature quantity having a strong correlation with a predetermined variation from the extracted feature quantities. Further, the image processing device 40 functions as an image generation device that learns the relationship between the extracted feature quantity and a process measurement value (process data) related to combustion, and generates processed image data (hereinafter referred to as feature image data) obtained by performing drawing processing on an image corresponding to the change in the feature quantity according to a predetermined variation.

[0053] The control unit 41 has a configuration that is functionally and physically the same as that of the control unit 31 described above. The control unit 41 includes a processor such as a CPU, DSP, or FPGA that has hardware, and a main memory unit (both not shown) such as a RAM or ROM. The output unit 42 as output means is configured to be able to notify external parties of predetermined information.

[0054] The output unit 42, in accordance with the control by the control unit 41, displays an image of the waste 50 in the incinerator 1 on a display monitor, displays characters, figures, etc. on the screen of a touch panel display, or outputs sound from a speaker. The input unit 43 as input means is configured using a user interface such as a keyboard, input buttons, a lever, a touch panel for manual input provided superimposed on a display such as a liquid crystal display, or a microphone for voice recognition. By a user or the like operating the input unit 43, it is configured to be able to input predetermined information to the control unit 41. Note that the output unit 42 and the input unit 43 may be integrated into an input / output unit, and the input / output unit may be configured from a touch panel display, a speaker microphone, etc.

[0055] The storage unit 44 has a configuration that is functionally and physically the same as that of the storage unit 32 described above, and is composed of a storage medium selected from a volatile memory such as a RAM, a non-volatile memory such as a ROM, an EPROM, an HDD, and a removable medium. The removable medium is, for example, a USB memory or a disk recording medium such as a CD, DVD, or BD. Also, the storage unit 44 may be configured using a computer-readable recording medium such as a memory card that can be externally attached.

[0056] The storage unit 44 can store an OS for executing the operations of the image processing apparatus 40, various programs, various tables, various databases, and the like. Here, the various programs include an information processing program for realizing control using the learning model or the learned model according to the present embodiment. The storage unit 44 may be provided in another server that can communicate via various networks, or may be provided in the combustion control device 30. Specifically, the storage unit 44 stores an image generation learning model 44a, a prediction learning model 44b, a feature amount extraction model 44c, and an image processing database 44d. The prediction learning model 44b stores an explainable artificial intelligence (XAI) (hereinafter, explainable AI 44e). The explainable AI 44e is applied to the prediction learning model 44b.

[0057] The image generation learning model 44a, the prediction learning model 44b, and the feature amount extraction model 44c are each updatable models including at least one learning model. When the learning model is not updated, it is stored in the storage unit 44 as a learned model. The image generation learning model 44a and the feature amount extraction model 44c may be constructed as one learning model, for example, an image feature extraction model. Further, an automatic determination processing program for realizing determination processing using a combustion image learning model capable of executing a predetermined determination from the combustion image data of the flame itself captured by the imaging unit 26 may be included. These various programs can also be recorded on a computer-readable recording medium such as a hard disk, a flash memory, a CD-ROM, a DVD-ROM, or a flexible disk and widely distributed.

[0058] The image generation learning model 44a is a learning model for generating images and is constructed from, for example, a GAN or the like. The prediction learning model 44b is a model that predicts changes after a predetermined time based on feature quantities extracted from transmission image data and process data. The feature quantity extraction model 44c is a model that extracts feature quantities from transmission image data and is constructed by, for example, a CNN or the like. Further, in the image generation learning model 44a and the feature quantity extraction model 44c, when using a VAE, it is possible to use an encoder and a decoder respectively. Also, the image generation learning model 44a, the prediction learning model 44b, and the feature quantity extraction model 44c may be configured as one learning model, such as one image processing learning model.

[0059] The image processing database 44d stores a plurality of transmission image data captured by the imaging unit 26, data of feature quantities extracted from the transmission image data, feature image data generated by the image processing apparatus 40, feature change image data described later, and information on the correlation between a predetermined change and the extracted feature quantities. Note that other data related to image processing may be stored.

[0060] The control unit 41 loads the program stored in the storage unit 44 into the work area of the main storage unit and executes it, and can realize a function that matches a predetermined purpose by controlling each component through the execution of the program. In the present embodiment, the control unit 41 executes the functions of the image processing unit 411, the feature quantity processing unit 412, and the learning unit 413 by executing the program stored in the storage unit 44. Specifically, for example, the control unit 41 executes the function of the image processing unit 411 by selectively reading a predetermined learning model from the image generation learning model 44a, the prediction learning model 44b, and the feature quantity extraction model 44c, which are programs, from the storage unit 44. Also, the control unit 41 executes the functions of the feature quantity processing unit 412 and the learning unit 413 for the prediction learning model 44b by reading the explainable AI 44e, which is a program, from the storage unit 44. Details of the functions of the image processing unit 411, the feature quantity processing unit 412, and the learning unit 413 will be described later.

[0061] (Image processing method) Next, the image processing method according to this embodiment will be described. FIG. 5 is a flowchart for explaining the image processing method according to this embodiment. As shown in FIG. 5, first, in step ST1, the imaging unit 26 continuously or intermittently images the waste being burned in the incinerator 1. The imaging of the inside of the incinerator 1 by the imaging unit 26 may be performed at a predetermined interval or continuously. When the imaging by the imaging unit 26 is performed continuously, the transmission image data becomes video data or moving image data.

[0062] The imaging unit 26 selectively outputs the captured transmission image data to the input unit 43 of the image processing apparatus 40 at predetermined time intervals, at any time, or in a timely manner. The control unit 41 stores the transmission image data input and acquired from the input unit 43 in the image processing database 44d of the storage unit 44. The transmission image data stored in the image processing database 44d is stored in time series in association with the date and time of imaging, that is, the information at the time of imaging. The acquisition of the transmission image data is performed at least in a number sufficient to generate a learning model and accumulated in the image processing database 44d. In this embodiment, for example, in a period of 7 to 10 days, preferably about 2 weeks, the transmission image data acquired at intervals of, for example, 1 minute is accumulated in the image processing database 44d. As a result, (60×24×7≒) 10,000 or more data of the transmission image data is accumulated.

[0063] Next, when shifting to step ST2, the image processing unit 411 of the control unit 41 extracts transmission image data that conforms to the change of the object or a predetermined change caused by the change of the object for a predetermined period. Specifically, for example, a predetermined change in which the influence caused by the combustion of the waste 50 appears is set as "the decrease in the furnace temperature which is the measured value by the lower gas thermometer 19 at the furnace outlet 10 minutes after a predetermined time point". In this case, the image processing unit 411 extracts a plurality of transmission image data for a predetermined time (hereinafter referred to as the retroactive time), for example, 5 minutes, retroactively from the plurality of transmission image data stored in the image processing database 44d starting from a predetermined time point. This retroactive time corresponds to a second retroactive time retroactively from a predetermined time point based on the information of the imaging time point associated with the transmission image data. Thereby, the image processing unit 411 can extract time-series transmission image data for the retroactive time from a predetermined time point when a predetermined change such as a decrease in the furnace temperature occurs after a predetermined elapsed time such as 10 minutes.

[0064] Subsequently, when shifting to step ST3, the learning unit 413 generates an image generation learning model 44a for converting from image data to a feature space. The image generation learning model 44a is a learning model such as a GAN, for example. The learning unit 413 stores the generated image generation learning model in the storage unit 44 and appends or updates the image generation learning model 44a. That is, the learning unit 413 of the control unit 41 generates an image generation learning model based on the image data captured inside the incinerator 1 and appends or updates the image generation learning model 44a. Also, in step ST3, the learning unit 413 generates a feature amount extraction model 44c for extracting feature amounts from the transmission image data. The feature amount extraction model 44c is constructed by, for example, a CNN. The learning unit 413 stores the generated feature amount extraction model in the storage unit 44 and appends or updates the feature amount extraction model 44c.

[0065] Next, in step ST4, the feature amount processing unit 412 of the control unit 41 extracts a feature amount as a multi-dimensional latent vector from a plurality of time-series transmission image data extracted by the image processing unit 411. In other words, the feature amount processing unit 412 quantifies the transmission image data as a feature amount. In the present embodiment, the feature amount is extracted by projecting it into a feature amount space learned by GAN using CNN. Further, the latent vector serving as the feature amount space is, for example, 16-dimensional, but it may be less than 16-dimensional or 17-dimensional or more. In image processing for transmission image data obtained by imaging the combustion of waste in the incinerator 1 as in the present embodiment, the feature amount space is preferably 16-dimensional, but is not limited thereto. If the number of dimensions of the latent vector to be extracted is less than 16-dimensional, the patterns of image data that can be generated or restored decrease, and only specific image data can be generated, and a so-called mode collapse phenomenon may occur. If the number of dimensions is 17-dimensional or more, the patterns of image data that can be generated or restored increase, and the feature image data becomes more detailed. Note that the number of dimensions of the latent vector to be extracted as the feature amount can be variously selected according to the object.

[0066] Next, shifting to step ST5, the image processing unit 411 of the control unit 41 acquires process data which is a process measurement value related to combustion in the incinerator 1 from the combustion control device 30. Note that the acquisition of the process data may be performed at a predetermined interval or continuously. Here, examples of the process data to be acquired include in-furnace temperature, in-furnace pressure, steam flow rate, steam pressure, air blowing amount, exhaust gas flow rate, pusher operation speed, fire grate operation speed, in-furnace camera information, and gas composition.

[0067] The image processing unit 411 stores the acquired process data in the image processing database 44d of the storage unit 44. The process data stored in the image processing database 44d is stored in chronological order in association with the measured date and time. Although the acquisition of the process data can be performed in the same manner as in the prior art, the image processing unit 411 acquires at least a number of process data sufficient to generate a predictive learning model and accumulates it in the image processing database 44d.

[0068] Next, the process proceeds to step ST6, where the image processing unit 411 extracts process data including a predetermined change from the process data stored in the image processing database 44d. Here, in the present embodiment, the process data to be acquired is various process data for a period of a predetermined retrospective time, for example, 5 minutes back from a predetermined time point 10 minutes before the above-described furnace temperature drop occurs. Here, the process data to be acquired is within a range traced back from a predetermined time point to a predetermined retrospective time before based on the measurement date and time associated with the process data. The retrospective time here is the first retrospective time traced back from the predetermined time point. Note that the retrospective time (the first retrospective time) from the predetermined time point for acquiring the process data is preferably the same as the retrospective time (the second retrospective time) for extracting the transmission image data, but it may be longer or shorter than the retrospective time for extracting the transmission image data. Thereby, the image processing unit 411 can acquire the process data for a retrospective time when a predetermined change such as a furnace temperature drop occurs after a predetermined elapsed time from the predetermined time point. Steps ST1 to ST6 described above may be executed in reverse order or in parallel. After the execution of steps ST1 to ST6, the process proceeds to step ST7.

[0069] In step ST7, the feature quantity processing unit 412 of the control unit 41 derives the correlation between the extracted feature quantity and a predetermined change, in this embodiment, the process data regarding the decrease in the furnace temperature after 10 minutes. Thereby, the predictive learning model 44b is generated or updated. Thereafter, the learning unit 413 applies the method of the explainable AI 44e to the predictive learning model 44b. Thereby, the feature quantity processing unit 412 extracts the relationship between the extracted feature quantity and the process data regarding the decrease in the furnace temperature after 10 minutes, for example, using SHAP, and calculates the contribution degree of each of the extracted, for example, 16-dimensional feature quantities to the prediction.

[0070] FIG. 6 is a graph showing the ease of occurrence of temperature decrease with respect to the values of the feature quantities selected by the feature quantity processing unit 412 of the image processing apparatus 40 according to this embodiment. The selected feature quantities shown in FIG. 6 are feature quantities with a high contribution degree to the decrease in the furnace temperature after 10 minutes among the 16-dimensional latent vectors. Here, the feature quantity with the highest contribution degree, that is, the feature quantity with the strongest correlation with the occurrence probability of temperature decrease is selected. As shown in FIG. 6, when the feature quantity is changed in the range of, for example, -2 to +2, as the feature quantity changes negatively, the ease of occurrence of temperature decrease increases rapidly. In other words, as the feature quantity changes positively, the ease of occurrence of temperature decrease decreases rapidly. The portion surrounded by the dotted line in FIG. 6 is the region where the furnace temperature corresponding to the feature quantity decreases. Here, in the feature quantity with a high contribution degree, since the relevance with the ease of occurrence of temperature decrease is high, when the value of the selected feature quantity is changed, the ease of occurrence of temperature decrease will change greatly. In this way, by extracting feature quantities with a high contribution degree to a predetermined change from among various feature quantities extracted from the transmission image data, feature quantities with a large correlation with the predetermined change can be selected.

[0071] Next, the process proceeds to step ST8, where the image processing unit 411 of the control unit 41 reads out the image generation learning model 44a and generates an image visualizing the factors of a predetermined change. That is, first, the image processing unit 411 derives the average value of the feature amounts such as the extracted 16-dimensional latent vector, for example, and inputs it as an input parameter to the image generation learning model 44a. The image generation learning model 44a generates and outputs, as output parameters, averaged feature image data based on the average value of the feature amounts.

[0072] FIG. 7 is a diagram showing an example of the average feature image data generated by the image processing apparatus 40. That is, FIG. 7 is the feature image data when the feature amount is set to 0 in the graph shown in FIG. 6. As shown in FIG. 7, the average feature image data is an image data generated in a state where the feature amounts are averaged, with the pre-feed waste 51 before being supplied onto the waste supply unit 12 and the fire grate 4, the step wall 13, the fire grate 4 and the waste 52 on the fire grate on the fire grate, and the furnace wall 1a, in a state where the flame (also referred to as a luminous flame) in the incinerator 1 is transparent. The image processing unit 411 stores the generated average feature image data in the image processing database 44d of the storage unit 44.

[0073] Thereafter, the image processing unit 411 generates feature change image data in which the feature amount that most contributes to the decrease in the furnace temperature after a predetermined elapsed time is changed. FIGS. 8 and 9 are diagrams showing examples of the feature change image data when the factor causing the furnace temperature to decrease and the factor causing the furnace temperature not to decrease are emphasized, respectively, with respect to the average feature image data shown in FIG. 7 generated by the image processing apparatus 40 according to the present embodiment.

[0074] When the selected feature amount is changed in the direction of decreasing temperature, here, the case where the selected feature amount shown in FIG. 6 is changed in the negative direction is taken as an example. In this case, as an input parameter, the feature amount necessary for generating image data equivalent to the transmission image data, including the feature amount in which the feature amount selected by the feature amount processing unit 412 is changed in the negative direction, is input to the image generation learning model 44a. The image generation learning model 44a outputs, as feature change image data output as an output parameter, for example, feature change image data as shown in FIG. 8 in which the layer height L1 of the waste 52A on the grate 4 is smaller than the averaged feature image data (see FIG. 7).

[0075] Further, the image processing unit 411 derives the luminance difference between the image data based on the feature amount in which the selected feature amount is changed in the negative direction and the averaged feature image data (see FIG. 7), so that the temperature T1 of the waste 51A before supply in the portion of the waste supply unit 12 is high, and for example, feature change image data in which a region with a high temperature T1 becomes red like a thermograph can be output. From the above, it is found that the factors causing the temperature to decrease after a predetermined elapsed time are the phenomenon that the layer height L1 of the waste 52A on the grate 4 is small and the temperature of the waste 51A before supply in the portion of the waste supply unit 12 is high, based on the feature change image data generated by changing the feature amount with a high contribution degree in the positive and negative directions.

[0076] On the contrary, when the selected feature quantity is changed in the direction in which the temperature does not decrease, here when the feature quantity shown in FIG. 6 is changed in the positive direction, as shown in FIG. 9, as the feature change image data, compared with the averaged feature image data (see FIG. 7), feature change image data in which the layer height L2 of the on-grate waste 52B on the fire grate 4 becomes slightly larger is obtained. Further, by deriving the luminance difference between the image data based on the feature quantity changed in the positive direction and the averaged feature image data (see FIG. 7), the temperature T2 of the pre-feed waste 51B in the part of the waste supply unit 12 is low, for example, feature change image data in which a region with a low temperature T2, such as in thermography, turns blue can be output. From the above points, as a factor for the temperature not decreasing after a predetermined elapsed time, it is found from the feature change image data generated by changing the feature quantity with a high contribution degree in positive and negative directions that the layer height L2 of the on-grate waste 52A on the fire grate 4 is slightly large and the temperature of the on-grate waste 52A in the part of the waste supply unit 12 is low.

[0077] The phenomenon that causes a predetermined change can be output as an image by the average feature image data obtained by averaging the feature quantities extracted from the transmission image data and the feature image data obtained by changing the feature quantity with a high contribution degree in positive and negative directions. As a result, the user who has recognized the image can recognize the factors and causes of a future predetermined change through the visualized image, enabling an intuitive recognition. Thus, the image processing according to the present embodiment is completed.

[0078] Thereafter, in step ST9, the image processing apparatus 40 transmits the generated feature image data and the information on the factors and causes obtained based on the feature image data to the combustion control apparatus 30. The control unit 31 of the combustion control apparatus 30 can control the incinerator 1 based on the received feature image data and the information on the factors of the predetermined change.

[0079] According to the above-described embodiment, a feature space quantified by a latent vector is generated using a plurality of transmission image data captured by the imaging unit 26, and a feature amount contributing to a predetermined change in the object is selected from the generated feature space and the process data in the incinerator 1, and the value of the selected feature amount is changed with respect to the averaged feature image data, and the feature image data is output. By deriving the difference from the averaged feature image data, the cause of a predetermined change can be output by an image. As a result, the cause of the temperature drop in the incinerator 1 can be recognized by an image. Therefore, it is possible to analyze which information among the information included in the captured image affects the situation of the object.

[0080] As described above, one embodiment of the present invention has been specifically described. However, the present invention is not limited to the above-described embodiment, and various modifications based on the technical idea of the present invention are possible. For example, the numerical values given in the above-described embodiment are merely examples, and different numerical values may be used as necessary. The present invention is not limited by the description and drawings that form a part of the disclosure of the present invention according to this embodiment.

[0081] For example, in the above-described embodiment, the image generation learning model 44a, the prediction learning model 44b, the feature amount extraction model 44c, the explainable AI 44e, and the image processing database 44d are stored in the storage unit 44, but it is also possible to store them in the storage unit of another server that can communicate through a network. That is, it is also possible to store the image generation learning model 44a, the prediction learning model 44b, and the feature amount extraction model 44c in the storage unit of an image server that can communicate with the image processing apparatus 40 via a network such as a public circuit network. In this case, the transmission image data captured by the imaging unit 26 is transmitted to the image server via the network and stored in the storage unit. Similarly, it is also possible to store the explainable AI 44e in the storage unit of a determination server that can communicate through a network.

[0082] Further, for example, an image learning unit having the functions of the image processing unit 411 and the learning unit 413, and the learning unit 413 and the feature amount processing unit 412 may be provided in separate devices that can communicate with each other via a network. Further, each of the image processing unit 411, the feature amount processing unit 412, and the learning unit 413 may be provided in separate devices that can communicate with each other via a network.

[0083] Also, for example, in the above-described embodiment, as examples of machine learning and artificial intelligence (AI), deep learning using a neural network, a machine learning framework that handles a gradient boosting algorithm based on a decision tree algorithm such as Light GBM, a so-called explainable artificial intelligence such as SHAP, and an adversarial generation network such as GAN which is a type of generative model are adopted, but machine learning based on other methods may also be performed. Further, semi-supervised learning or supervised learning may be used instead of unsupervised learning.

[0084] In the above-described embodiment, a waste incinerator is used as the target facility, waste is used as the target object, and the decrease in the furnace temperature after a predetermined elapsed time is used as an example of the predetermined change. However, the above-described embodiment is applicable to other target facilities, target objects, and predetermined changes. Specifically, as target facilities for the application field, in addition to waste incinerators, combustion power plants (combustion power generation facilities), water treatment plants (water treatment facilities), powder storage plants (powder storage facilities) that store powder, and manufacturing plants (manufacturing facilities) that manufacture predetermined products can be adopted.

[0085] When the target facility is a waste incinerator, the objects can include waste (garbage), incineration ash, etc. When the object is waste, the predetermined change can be fluctuations in the quality of the waste, i.e., physical properties such as calorific value and ease of movement. In this case, the image processing device 40 can visualize or identify factors such as the calorific value and ease of movement, which are physical properties resulting from fluctuations in the quality of the waste. Also, when the object is incineration ash, the predetermined change can be fluctuations in the properties of the incineration ash. In this case, the image processing device 40 can visualize or identify the factors causing fluctuations in the properties of the incineration ash.

[0086] When the target facility is a combustion power plant, it includes a combustion power plant installed in parallel with a waste incinerator, etc., and the objects can include waste, etc. When the object is waste, the predetermined change can be fluctuations in temperature, concentration fluctuations of nitrogen oxide (NO x ), and concentration fluctuations of carbon monoxide (CO), etc. In this case, the image processing device 40 can visualize or identify factors such as fluctuations in temperature, NO x concentration fluctuations, and CO concentration fluctuations in the combustion power plant. Similarly, the factors causing fluctuations in the power generation amount can be visualized or identified.

[0087] When the target facility is a water treatment plant, the object can be treated water, and as image data, images of water tanks or microscopic images during water treatment can be used. When the object is treated water, the predetermined change can be fluctuations in the quality of the treated water observable in images of water tanks or microscopic images, etc. In this case, the image processing device 40 can visualize or identify the factors causing fluctuations in the quality of the treated water from images of water tanks or microscopic images, etc. in the water treatment plant.

[0088] Also, when the target facility is a powder storage plant, the object can be the stored powder, and the image data can be the captured image of the inside of the powder storage section. When the object is powder, the predetermined change can be, for example, a change in the level of the stored powder. In this case, the image processing apparatus 40 can visualize or identify factors such as fluctuations in the storage level of the powder in the powder storage plant.

[0089] Also, when the target facility is a manufacturing plant for a predetermined product, the object can be the predetermined product, and the image data can be the image of the assembled product during manufacturing. When the object is a predetermined product, the predetermined change can be the product quality, etc. In this case, the image processing apparatus 40 can visualize or identify factors such as fluctuations in the product quality, that is, whether it is a good product or a defective product, from the image of the assembled product during manufacturing in the manufacturing plant.

[0090] (Recording medium) In the above-described embodiment, a program for causing the combustion control device 30 or the image processing device 40 to execute the processing method can be recorded on a recording medium readable by a computer or other devices such as machines and wearable devices (hereinafter referred to as a computer or the like). By causing a computer or the like to read and execute the program of this recording medium, the computer or the like functions as a movement control device. Here, a recording medium readable by a computer or the like refers to a non-temporary recording medium that accumulates information such as data and programs by an electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer or the like. Examples of removable recording media among such recording media include flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs, BDs, DATs, magnetic tapes, memory cards such as flash memories, etc. Also, as recording media fixed to a computer or the like, there are hard disks, ROMs, etc. Furthermore, an SSD can be used as both a removable recording medium from a computer or the like and a recording medium fixed to a computer or the like.

[0091] Also, the program to be executed by the combustion control device 30 and the image processing device 40 according to an embodiment may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0092] (Other Embodiments) In one embodiment, the above-described "section" can be read as "circuit" or the like. For example, the control section can be read as a control circuit.

[0093] In the description of the flowchart in this specification, expressions such as "first", "next", "then", and "subsequently" are used to clarify the sequence of processing between steps. However, the order of processing necessary to implement this embodiment is not uniquely determined by these expressions. That is, the order of processing in the flowchart described in this specification can be changed within a non - contradictory range.

[0094] Further effects and modifications can be easily derived by those skilled in the art. The broader aspects of the present disclosure are not limited to the specific details and representative embodiments presented and described as above. Therefore, various changes can be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

Explanation of Reference Numerals

[0095] 1 Incinerator 1a Furnace wall 2 Waste inlet 3 Waste supply device 4 Grate 5 Ash outlet 6 Combustion air blower 7 Furnace outlet 8 Chimney 9 Boiler 9a Heat exchanger 9b Steam drum 10 Secondary air inlet 11 Secondary air blower 12 Waste supply section 13 Step wall 14 Combustion air damper 14a, 14b, 14c, 14d Under - grate combustion air dampers 15 Secondary air damper 16 Intermediate ceiling 17 Combustion chamber gas thermometer 18 Main flue gas thermometer 19 Lower furnace outlet gas thermometer 20 Middle furnace outlet gas thermometer 21 Furnace outlet gas thermometer 22 Boiler outlet oxygen concentration meter 23 Gas concentration meter 24 Exhaust gas flow meter 25 Steam flow meter 26 Imaging unit 30 Combustion control device 31, 41 Control unit 32, 44 Memory unit 33 Manipulation quantity adjustment unit 40 Image processing device 42 Output unit 43 Input unit 44a Image generation learning model 44b Prediction learning model 44c Feature extraction model 44d Image processing database 44e Explainable AI 50 Waste 51, 51A, 51B Waste before supply 52, 52A, 52B Waste on the grate 331 Combustion air quantity adjustment unit 332 Air quantity ratio adjustment unit 333 Secondary air quantity adjustment unit 334 Waste supply device feed speed adjustment unit 335 Grate feed speed adjustment unit 411 Image processing unit 412 Feature quantity processing unit 413 Learning unit

Claims

1. A control unit having hardware, wherein the control unit acquires environmental data related to the environment in which the object exists for predicting the occurrence of a change in the object or a predetermined change resulting from the change in the object, and stores the environmental data in a storage unit; reads out from the storage unit the environmental data related to the change in the object or the predetermined change resulting from the change in the object at a time after a predetermined elapsed time from a predetermined time point within a range retroactively from the predetermined time point to a predetermined first retroactive time; acquires imaging image data generated by imaging the object by an imaging unit, and stores the imaging image data in the storage unit in association with information on the imaging time point; reads out a plurality of the imaging image data from the storage unit within a range retroactively from the predetermined time point to a predetermined second retroactive time; extracts a plurality of quantified feature amounts from each of the read plurality of imaging image data and stores the feature amounts in the storage unit; selects at least one feature amount having a large correlation with the change in the object or a predetermined change resulting from the change in the object among the plurality of feature amounts; inputs the extracted feature amounts into an image generation learning model for generating image data to generate feature image data, and changes the feature image data by changing numerical values of the selected feature amounts to generate and output feature change image data. An information processing apparatus.

2. The control unit inputs, as input parameters, feature amounts obtained by averaging the plurality of quantified feature amounts into the image generation learning model, and causes the image generation learning model to output, as output parameters, averaged feature image data. The information processing apparatus according to claim 1.

3. The control unit derives a difference between the averaged feature image data and the feature change image data, and extracts a factor of the predetermined change in the feature change image data. The information processing apparatus according to claim 2.

4. The difference is a difference in luminance. The information processing apparatus according to claim 3.

5. The object is waste in an incinerator, and the environmental data in the environment where the object exists is process data in the incinerator. The information processing apparatus according to claim 1.

6. A predetermined change resulting from the change in the object is a change in the temperature inside the incinerator or a change in the concentration of nitrogen oxide or carbon monoxide discharged from the incinerator. The information processing apparatus according to claim 5.

7. The first retroactive time and the second retroactive time are the same time. The information processing apparatus according to claim 1.

8. An information processing method executed by an information processing apparatus including a control unit having hardware, acquiring environmental data related to the environment in which the object exists for predicting the occurrence of a change in the object or a predetermined change caused by the change in the object, and storing the environmental data in a storage unit, reading out, from the storage unit, the environmental data related to the predetermined change that occurs in the object after a predetermined elapsed time from a predetermined time, within a range retroactively from the predetermined time to a predetermined first retroactively time before, acquiring imaging image data generated by imaging the object by an imaging unit, and storing the imaging image data in the storage unit in association with information at the imaging time, reading out a plurality of the imaging image data from the storage unit within a range retroactively from the predetermined time to a predetermined second retroactively time before, extracting a plurality of quantified feature amounts from each of the read plurality of imaging image data and storing the feature amounts in the storage unit, selecting at least one feature amount having a large correlation with the change in the object or a predetermined change caused by the change in the object among the plurality of feature amounts, inputting the extracted feature amounts into an image generation learning model that generates image data to generate feature image data, and changing the numerical values of the selected feature amounts to generate feature change image data obtained by changing the feature image data Information processing method.

9. In the control unit in an information processing apparatus including a control unit having hardware, acquiring environmental data related to the environment in which the object exists for predicting the occurrence of a change in the object or a predetermined change caused by the change in the object, and storing the environmental data in a storage unit, reading out, from the storage unit, the environmental data related to the predetermined change that occurs in the object after a predetermined elapsed time from a predetermined time, within a range retroactively from the predetermined time to a predetermined first retroactively time before, acquiring imaging image data generated by imaging the object by an imaging unit, and storing the imaging image data in the storage unit in association with information at the imaging time, reading out a plurality of the imaging image data from the storage unit within a range retroactively from the predetermined time to a predetermined second retroactively time before, extracting a plurality of quantified feature amounts from each of the read plurality of imaging image data and storing the feature amounts in the storage unit, selecting at least one feature amount having a large correlation with the change in the object or a predetermined change caused by the change in the object among the plurality of feature amounts, A program that causes an image generation learning model for generating image data to input the extracted feature amounts to generate feature image data, and to change the numerical values of the selected feature amounts to generate feature change image data obtained by changing the feature image data. A program to execute this.

Images