Computer program, information processing method and information processing apparatus

Abstract

To provide a computer program, an information processing method and an information processing apparatus that are expected to easily and accurately determine the conditions of plural components such as blocks, fire grates or the like configuring the inside face of a combustion chamber in a burning furnace.SOLUTION: A computer program according to this embodiment causes a computer to execute the steps of: acquiring a captured image of an inside face of a combustion chamber; detecting plural components configuring the inside face of the combustion chamber from the captured image; determining the conditions of the detected respective components; and outputting condition information that associates the layout of the plural components with the conditions of the respective components.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a computer program, an information processing method, and an information processing device for determining the state of a plurality of components provided on an inner surface of a combustion chamber. [Background technology]

[0002] Patent document 1 proposes an inspection device for a stoker furnace, which includes a plurality of vibration-damping wires whose upper ends are connected to a fixed plate fixed to the underside of the furnace ceiling in the furnace space, a lifting stand whose vibration is damped by the plurality of vibration-damping wires, a measuring instrument arranged on the lifting stand, and a lifting device for raising and lowering the lifting stand. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-116237 A Summary of the Invention [Problem to be solved by the invention]

[0004] For example, the wall surface of the combustion chamber of an incinerator is made up of a number of blocks arranged side by side, and these blocks must be inspected periodically to see if they have any abnormalities such as cracks or chips. Also, for example, a number of fire grates are arranged side by side on the bottom surface of the combustion chamber, and these fire grates must be inspected periodically to see if they have any abnormalities such as wear. Conventionally, these inspection tasks have been performed visually by workers.

[0005] The present disclosure has been made in consideration of the above circumstances, and its purpose is to provide a computer program, an information processing method, and an information processing device that can be expected to easily and accurately determine the condition of multiple components, such as blocks or grates, that make up the inner surface of an incinerator combustion chamber. [Means for solving the problem]

[0006] A computer program according to one embodiment causes a computer to execute a process of acquiring a photographed image of the interior of a combustion chamber, detecting a plurality of components that make up the interior of the combustion chamber from the photographed image, determining the state of each of the detected components, and outputting state information that associates the arrangement of the plurality of components with the state of each component. Effect of the Invention

[0007] According to the present disclosure, it is expected that the condition of multiple components that make up the inner surface of the combustion chamber of an incinerator can be determined easily and accurately. [Brief description of the drawings]

[0008] [Figure 1] 1 is a schematic diagram for explaining an overview of an information processing system according to an embodiment of the present invention; [Diagram 2] 1 is a block diagram showing an example of a configuration of an information processing device according to an embodiment of the present invention; [Diagram 3] 2 is a block diagram showing a configuration of a terminal device according to the present embodiment; FIG. [Figure 4] 5 is a schematic diagram for explaining a state determination of a block on the inner surface of a combustion chamber performed by an information processing device according to the present embodiment; FIG. [Diagram 5] FIG. 2 is a schematic diagram showing an example of a configuration of a learning model used by the information processing device. [Figure 6] FIG. 11 is a schematic diagram showing an example of state information displayed by a terminal device. [Figure 7] 10 is a flowchart showing an example of a procedure of a state determination process performed by the information processing device according to the present embodiment. [Figure 8] FIG. 13 is a schematic diagram showing an example of detection of a grate. [Figure 9] FIG. 11 is a schematic diagram showing an example of a configuration of a learning model used by an information processing device according to embodiment 3. [Figure 10] 13 is a flowchart showing an example of a procedure of a state determination process performed by the information processing device 1 according to the embodiment 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Specific examples of information processing systems according to embodiments of the present disclosure will be described below with reference to the drawings. Note that the present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims.

[0010] <System Overview> 1 is a schematic diagram for explaining the outline of an information processing system according to this embodiment. The information processing system according to this embodiment is a system that judges the state of a plurality of components constituting the inner surface (wall surface, ceiling surface, bottom surface, etc.) of a combustion chamber of an incinerator such as a stoker type incinerator, and provides a user with state information of each component. As an example, in this embodiment, it judges whether or not there is an abnormality such as a chip or crack for a plurality of blocks constituting the wall surface of the combustion chamber, and provides a user with state information that corresponds to the position of the abnormal block and its state.

[0011] A user photographs the interior of the combustion chamber using a terminal device 3, such as a smartphone or tablet terminal device equipped with a camera, at an appropriate time, such as a regular inspection of an incinerator. The terminal device 3 transmits data of the photographed image of the interior of the combustion chamber obtained by photographing with the camera to an information processing device 1, such as a server device. The user may photograph the interior of the combustion chamber at any time. The data obtained by photographing with the terminal device 3 may be either a still image or a moving image.

[0012] The information processing device 1 according to this embodiment receives photographed image data of the combustion chamber inner surface from the terminal device 3, and stores the received photographed image data in, for example, a database. The information processing device 1 also detects multiple blocks constituting the wall surface of the combustion furnace shown in the photographed image based on the photographed image data acquired from the terminal device 3. The information processing device 1 individually determines the presence or absence of abnormalities such as chips or cracks for the multiple blocks detected from the photographed image. The information processing device 1 transmits information related to the results of the state determination for each block to the terminal device 3, and provides this information to the user by displaying it on the terminal device 3.

[0013] In this embodiment, the information provided to the user by the information processing device 1 can be, for example, an image in which information such as characters, colors, or images indicating the state of each block is arranged in a matrix when multiple blocks constituting the inner surface of the incinerator are stacked in a matrix. In this image of the state determination result, the information on the state determination result of each block is arranged in correspondence with the actual arrangement of the multiple blocks constituting the inner surface of the incinerator combustion chamber. By providing such information to the user, it is expected that the user can confirm, for example, in what places on the inner surface of the incinerator combustion chamber anomalies of blocks are occurring or are likely to occur.

[0014] In this embodiment, the user photographs the inside of the combustion chamber of the incinerator using the terminal device 3, and the information on the state of each block determined by the information processing device 1 based on the photographed image is displayed and provided to the user by the terminal device 3, but the information may be provided to the user by the information processing device 1 instead of the terminal device 3. In this case, the user can use the terminal device 3 as an imaging device and perform tasks such as checking and verifying the state information of each block based on the photographed image on the information processing device 1.

[0015] Furthermore, for example, processing such as detection of multiple blocks based on a captured image or determination of the state of each block may be performed by the terminal device 3 rather than the information processing device 1. Information processing such as photographing the interior of the combustion chamber of the incinerator, detecting blocks from the captured image, determining the state of each block, and displaying information related to the determination results may all be performed by the terminal device 3 alone, in which case the information processing system does not need to include the information processing device 1.

[0016] <Device configuration> 2 is a block diagram showing an example of the configuration of an information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment is configured to include a processing unit 11, a memory unit (storage) 12, and a communication unit (transceiver) 13. Note that, in this embodiment, the description will be given assuming that processing is performed by one information processing device 1, but processing may be distributed among a plurality of information processing devices.

[0017] The processing unit 11 is configured using an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit) or a quantum processor, and a storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processing unit 11 reads out and executes a program 12a stored in the storage unit 12, thereby performing various processes such as a process of detecting multiple blocks constituting the inner surface of the combustion chamber based on the data of the photographed image of the inner surface of the combustion chamber of the incinerator obtained by photographing using the terminal device 3, and a process of determining the state of each detected block.

[0018] The storage unit 12 is configured using a large-capacity storage device such as a hard disk. The storage unit 12 stores various programs executed by the processing unit 11 and various data required for the processing of the processing unit 11. In this embodiment, the storage unit 12 stores a program 12a executed by the processing unit 11. The storage unit 12 also includes a state information storage unit 12b that stores information regarding the state of a block determined based on a photographed image of the combustion chamber inner surface acquired from the terminal device 3.

[0019] In this embodiment, the program (computer program, program product) 12a is provided in a form recorded on a recording medium 99 such as a memory card or an optical disk, and the information processing device 1 reads out the program 12a from the recording medium 99 and stores it in the storage unit 12. However, the program 12a may be written in the storage unit 12, for example, during the manufacturing stage of the information processing device 1. Also, for example, the program 12a may be distributed by another remote server device or the like and acquired by the information processing device 1 through communication. For example, the program 12a may be read from the recording medium 99 by a writing device and written into the storage unit 12 of the information processing device 1. The program 12a may be provided in a form distributed via a network, or may be provided in a form recorded on the recording medium 99.

[0020] The status information storage unit 12b stores, in association with each other, a photographed image of the interior of the combustion chamber obtained, for example, from the terminal device 3, associated information such as the date and time of the photograph and the identification information of the photographer, and information regarding the status of the block determined by the information processing device 1 based on the photographed image.

[0021] The communication unit 13 communicates with various devices via a network N including a mobile phone communication network, a wireless LAN (Local Area Network), the Internet, etc. In this embodiment, the communication unit 13 communicates with one or more terminal devices 3 via the network N. The communication unit 13 transmits data provided from the processing unit 11 to other devices, and provides data received from other devices to the processing unit 11.

[0022] The storage unit 12 may be an external storage device connected to the information processing device 1. The information processing device 1 may be a multi-computer including a plurality of computers, or may be a virtual machine virtually constructed by software. The information processing device 1 is not limited to the above configuration, and may include, for example, a reading unit that reads information stored in a portable storage medium, an input unit that accepts operation input, or a display unit that displays an image.

[0023] In the information processing device 1 according to this embodiment, the processing unit 11 reads out and executes the program 12a stored in the storage unit 12, whereby a data acquisition unit 11a, a component detection unit 11b, a state determination unit 11c, an information transmission processing unit 11d, and the like are realized as software functional units in the processing unit 11. Note that in this drawing, functional units related to the processing of determining the states of a plurality of blocks constituting the inner surface of the combustion chamber are shown as the functional units of the processing unit 11, and functional units related to other processes are not shown.

[0024] The data acquisition unit 11a performs a process of acquiring data of a photographed image of the interior of the combustion chamber photographed by the terminal device 3, by communicating with the terminal device 3 used by the user via the communication unit 13. The data acquisition unit 11a acquires the photographed image data by receiving the photographed image data transmitted by the terminal device 3 via the communication unit 13, and stores the acquired photographed image in the status information storage unit 12b. At this time, the data acquisition unit 11a may acquire information such as the measurement date and time and identification information of the person measuring together with the photographed image data from the terminal device 3, and store this information in the status information storage unit 12b in association with the photographed image data.

[0025] The component detection unit 11b performs a process of detecting a plurality of blocks constituting the combustion chamber inner surface captured in the captured image based on the data of the captured image of the combustion chamber inner surface acquired by the data acquisition unit 11a from the terminal device 3. In this embodiment, the wall of the incinerator is constructed by stacking a plurality of blocks. Each block is, for example, approximately cubic, and in the captured image of the combustion chamber inner surface, one face of each cubic block is captured in a matrix arrangement. For example, the component detection unit 11b performs image processing such as edge extraction on the captured image to extract the boundary line between the blocks from the captured image, and detects rectangular areas such as squares or rectangles surrounded by the boundary line, thereby detecting a plurality of blocks from the captured image. In addition, for example, the component detection unit 11b may perform a process of detecting a plurality of blocks from the captured image by using a detection model that has previously learned the shape and color of the blocks by machine learning.

[0026] The state determination unit 11c performs a process of determining the presence or absence of abnormalities such as chipping or cracking for each of the multiple blocks detected by the component detection unit 11b from the captured image. In this embodiment, the state determination unit 11c performs state determination for each block using a learning model that has been machine-learned in advance. The learning model is machine-learned in advance to receive, for example, an image in which a block is captured as an input, and output information indicating the state of the block captured in this image. For example, a learning model having a configuration such as CNN (Convolutional Neural Network) can be adopted as the learning model. The learning model is generated in advance by supervised machine learning using, for example, learning data (teacher data) in which an image in which a block is captured corresponds to the state of the block. The state determination unit 11c can determine the state of each block by extracting image areas in which each block is captured from the captured image, inputting the extracted image areas to the learning model, and acquiring information output by the learning model. In this embodiment, the state of each block determined by the state determination unit 11c may be various states, such as a normal state, a state with a chipped corner, a state with a crack (such as a fissure), a state with a linear crack, or a state with part of the surface missing, etc. The state determination unit 11c stores information on the result of the state determination of the blocks constituting the inner surface of the combustion chamber based on the photographed image in the state information storage unit 12b in association with the photographed image used for the determination.

[0027] The information transmission processing unit 11d performs a process of transmitting information about the state of each block determined by the state determination unit 11c to the terminal device 3 that transmitted the photographed image of the combustion chamber inner surface. In this embodiment, the information transmission processing unit 11d grasps the position of each block based on the detection result of the component detection unit 11b detecting blocks from the photographed image, and grasps the state of each block based on the determination result of the state determination unit 11c. The information transmission processing unit 11d, for example, arranges a plurality of squares corresponding to the arrangement of the plurality of blocks detected from the photographed image, generates an image in which the state of each corresponding block is indicated in each square by characters, colors, or the like, and transmits the data of this image to the terminal device 3 as information about the result of the state determination.

[0028] FIG. 3 is a block diagram showing the configuration of the terminal device 3 according to this embodiment. The terminal device 3 according to this embodiment is configured to include a processing unit 31, a memory unit (storage) 32, a communication unit (transceiver) 33, a display unit (display) 34, an operation unit 35, a camera 36, ​​and a distance measuring sensor 37. The terminal device 3 is a device used by a user such as an operator who inspects the combustion chamber or an administrator who manages the incinerator, and may be configured using an information processing device such as a smartphone or a tablet terminal device. In the information processing system according to this embodiment, the user uses the terminal device 3 to take an image of the inside of the combustion chamber of the incinerator, but this is not limited to this. For example, the user may take an image using a photographing device such as a digital camera or a digital video camera, and the obtained photographed image may be provided to the information processing device 1 via communication or a recording medium.

[0029] The processing unit 31 is configured using an arithmetic processing device such as a CPU or an MPU, a ROM, etc. The processing unit 31 reads out and executes a program 32a stored in the storage unit 32, thereby performing various processes such as photographing the interior of the combustion chamber and displaying the results of judgment of the state of the block based on the photographed image.

[0030] The storage unit 32 is configured using a non-volatile memory element such as a flash memory or a storage device such as a hard disk. The storage unit 32 stores various programs executed by the processing unit 31 and various data required for the processing of the processing unit 31. In this embodiment, the storage unit 32 stores the program 32a executed by the processing unit 31. In this embodiment, the program (computer program, program product) 32a is distributed by a remote server device or the like, and the terminal device 3 acquires it through communication and stores it in the storage unit 32. However, the program 32a may be written in the storage unit 32, for example, during the manufacturing stage of the terminal device 3. For example, the program 32a may be read by the terminal device 3 from the program 32a recorded in a recording medium 98 such as a memory card or an optical disk and stored in the storage unit 32. For example, the program 32a may be read by a writing device from the program 32a recorded in the recording medium 98 and written in the storage unit 32 of the terminal device 3. The program 32a may be provided in a form of distribution via a network, or in a form of being recorded in the recording medium 98.

[0031] The communication unit 33 communicates with various devices via a network N including a mobile phone communication network, a wireless LAN, the Internet, etc. In this embodiment, the communication unit 33 communicates with the information processing device 1 via the network N. The communication unit 33 transmits data provided from the processing unit 31 to other devices, and provides data received from other devices to the processing unit 31.

[0032] The display unit 34 is configured using a liquid crystal display or the like, and displays various images, characters, and the like based on the processing of the processing unit 31. The operation unit 35 accepts user operations and notifies the processing unit 31 of the accepted operations. For example, the operation unit 35 accepts user operations through input devices such as mechanical buttons or a touch panel provided on the surface of the display unit 34. Furthermore, for example, the operation unit 35 may be input devices such as a mouse and a keyboard, and these input devices may be configured to be detachable from the terminal device 3.

[0033] The camera 36 is configured using an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 36 is provided in the housing of the terminal device 3, and provides data of an image captured by the imaging element to the processing unit 31.

[0034] The distance measurement sensor 37 is, for example, a sensor such as LiDAR (Light Detection And Ranging) that measures the distance to an object by irradiating laser light and measuring the reflected light. Note that the distance measurement sensor 37 is not limited to a sensor that measures distance using laser light, and may be a sensor that measures distance using, for example, radio waves, ultrasonic waves, or infrared light. The distance measurement sensor 37 provides data on the measured distance to the processing unit 31. Note that in this embodiment, the terminal device 3 does not need to include the distance measurement sensor 37 (the distance measurement sensor 37 is used in embodiment 3).

[0035] The terminal device 3 simultaneously captures the image of the combustion chamber interior with the camera 36 and measures the distance with the distance measuring sensor 37, and generates three-dimensional shape data in which color (R, G, B) information and three-dimensional coordinate (X, Y, Z) information are given for each pixel of the captured image, based on the image data obtained by capturing the image and the distance data obtained by measuring the distance. The camera 36 and the distance measuring sensor 37 may be provided in the terminal device 3 as a single unit.

[0036] In addition, in the terminal device 3 according to this embodiment, the processing unit 31 reads out and executes the program 32a stored in the memory unit 32, so that the data transmission processing unit 31a, the display processing unit 31b, etc. are realized in the processing unit 31 as software functional units.

[0037] The data transmission processing unit 31a performs a process of transmitting data of an image captured by the camera 36 to the information processing device 1 via the communication unit 33. At this time, the data transmission processing unit 31a may transmit information such as the date and time when the image was captured and the identification information of the user who captured the image, to the information processing device 1, with respect to the data to be transmitted. Furthermore, the data transmission processing unit 31a may transmit data on the measurement result of the distance to the inner surface of the combustion chamber by the distance measuring sensor 37, which was performed simultaneously with the image capture by the camera 36, ​​to the information processing device 1 together with the image data obtained by the image capture.

[0038] The display processing unit 31b performs processing to display various characters, images, and the like on the display unit 34. The display processing unit 31b can display, for example, an image captured by the camera 36 on the display unit 34. In the present embodiment, the display processing unit 31b receives information on the state determination result of the blocks on the inner surface of the combustion chamber determined by the information processing device 1, and displays the state determination result of the blocks on the inner surface of the combustion chamber on the display unit 34 based on the received information. As described above, in the information processing system according to the present embodiment, the information processing device 1 generates an image showing the determination result of the state of each block in a plurality of squares corresponding to a plurality of blocks detected from the captured image and transmits it to the terminal device 3, and the display processing unit 31b of the terminal device 3 can receive this image and display it on the display unit 34.

[0039] <Status Judgment Process> Fig. 4 is a schematic diagram for explaining the state judgment of the block on the combustion chamber inner surface performed by the information processing device 1 according to this embodiment. The upper part of Fig. 4 shows an example of a photographed image of the combustion chamber inner surface photographed by the terminal device 3. The information processing device 1 acquires data of the photographed image of the combustion chamber inner surface from the terminal device 3, and stores the acquired photographed image data in the state information storage unit 12b together with information such as the date and time of the photographed image and the identification information of the photographer.

[0040] The information processing device 1 also performs a process of detecting a plurality of blocks constituting the combustion chamber inner surface captured in the captured image based on the captured image data acquired from the terminal device 3. The middle part of FIG. 4 shows an example of the result of block detection for the captured image in the upper part. In this embodiment, the combustion chamber inner surface of the incinerator is composed of a plurality of stacked cubic blocks, and in the captured image of the combustion chamber inner surface, a rectangular side of each block is captured in a matrix arrangement. The information processing device 1 extracts boundary lines separating the plurality of blocks arranged in a matrix by performing image processing such as edge extraction on the captured image. The image in the middle part of FIG. 4 shows these vertical and horizontal boundary lines superimposed on the captured image. The information processing device 1 detects a rectangular area surrounded by these vertical and horizontal boundary lines as one block.

[0041] The information processing device 1 performs a process of determining whether each block detected from the captured image corresponds to a state such as normal, missing surface parts, chipped corners, cracks, etc. In this embodiment, the information processing device 1 receives, for example, an image in which blocks are captured as an input, and determines the states of the multiple blocks detected from the captured image using a learning model that has been machine-learned in advance to output information indicating the states of the blocks captured in the image.

[0042] 5 is a schematic diagram showing an example of a configuration of a learning model used by the information processing device 1. The learning model used by the information processing device 1 according to the present embodiment is generated in advance by machine learning as a classification model that accepts, for example, data of a captured image as an input and classifies a block captured in the captured image into one of a number of states, such as normal, a missing surface portion, a chipped corner, or a crack. For example, the learning model outputs one numerical value for each of a number of states to be classified, and the state with the largest numerical value becomes the classification result for the input image data.

[0043] The learning model is, for example, a learning model configured as a CNN or the like, and the information processing device 1 generates in advance a learning model that classifies the state of a block by performing so-called supervised machine learning using learning data that associates an image in which a block is captured with the state of the block. Note that the machine learning process for generating the learning model may be performed by a device different from the information processing device 1. The information processing device 1 stores in advance in the storage unit 12 information indicating the structure of the learning model and information such as internal parameters obtained by machine learning, and can reproduce the learning model based on this information when determining the state of the block.

[0044] The bottom part of Fig. 4 shows an example of the result of judging the state of each block based on the block detection result in the middle part. In this example, a numerical value superimposed on each block shown in the captured image indicates the state judgment result of that block. For example, the numerical value 0 corresponds to a normal state, and for example, the numerical value 4 corresponds to a surface part falling off, and each numerical value corresponds to one of the states classified by the learning model.

[0045] In this embodiment, the information processing device 1 extracts an image area in which each block is captured from a captured image, inputs the extracted image area to a learning model, obtains multiple values ​​output by the learning model in response to this input, and determines the state of the block as the state corresponding to the maximum value among the multiple obtained values. The information processing device 1 repeatedly performs the same process for multiple blocks detected from the captured image, and determines the state of all blocks detected from the captured image.

[0046] After completing the state determination for all blocks detected from the captured image, the information processing device 1 generates state information that associates the arrangement of the multiple blocks that make up the combustion chamber inner surface with the state of each block. The information processing device 1 stores the generated state information in the state information storage unit 12b in association with the captured image, and transmits it to the terminal device 3 that transmitted the captured image. The terminal device 3 receives the state information transmitted from the information processing device 1 and performs a display based on the received state information, thereby providing the user with information regarding the state of each block on the combustion chamber inner surface.

[0047] FIG. 6 is a schematic diagram showing an example of display of status information by the terminal device 3. The terminal device 3 according to the present embodiment displays, on the display unit 34, an image in which a plurality of rectangles imitating the shapes of blocks are arranged in the same arrangement as a plurality of blocks detected from a photographed image, and each rectangle is color-coded according to the state of the corresponding block (however, in FIG. 6, the color coding is expressed by the type of hatching). In the example shown in FIG. 6, the state of each block is classified into four states: normal, surface part falling off, chipped corners, and cracks, and the terminal device 3 displays the color coding corresponding to each state as a legend. The terminal device 3 also displays information about the date when the image of the inside of the combustion chamber of the incinerator was taken and the photographer.

[0048] 6 may be generated by the information processing device 1 based on the result of the state determination, for example, and the information processing device 1 may transmit the generated image to the terminal device 3 as state information, and the terminal device 3 may display the received image on the display unit 34. Alternatively, for example, the information processing device 1 may transmit information associating the arrangement of a plurality of blocks with the state of each block to the terminal device 3 as state information, and the terminal device 3 may receive the information and generate the image shown in FIG.

[0049] 7 is a flow chart showing an example of the procedure of the state determination process performed by the information processing device 1 according to this embodiment. The data acquisition unit 11a of the processing unit 11 of the information processing device 1 according to this embodiment communicates with the terminal device 3 via the communication unit 13, and acquires data of a photographed image of the interior of the combustion chamber of the incinerator photographed by the camera 36 of the terminal device 3 (step S1). At this time, the data acquisition unit 11a also acquires information such as the photographing date and time of the photographed image and the identification information of the photographer from the terminal device 3, and stores the data of the photographed image in the state information storage unit 12b in association with this information (step S2).

[0050] Next, the component detection unit 11b of the processing unit 11 performs processing to detect multiple blocks constituting the combustion chamber inner surface captured in the captured image based on the data of the captured image acquired in step S1 (step S3). At this time, the component detection unit 11b performs image processing such as edge extraction on the captured image to extract boundaries separating multiple blocks arranged in a matrix, and detects a rectangular area surrounded by vertical and horizontal boundaries as one block.

[0051] Next, the state determination unit 11c of the processing unit 11 extracts an image area in which one of the blocks detected from the captured image is captured based on the detection result of step S3 (step S4). The state determination unit 11c inputs the image area of ​​the block extracted in step S4 to a learning model that has been machine-learned in advance (step S5). The state determination unit 11c acquires information output by the learning model in response to the input of the image in step S5 (step S6). The state determination unit 11c can determine the state of the block based on the information output by the learning model.

[0052] The state determination unit 11c determines whether the processes of steps S4 to S6 have been completed for all blocks detected in step S3 (step S7). If the processes have not been completed for all blocks (S7: NO), the state determination unit 11c returns the process to step S4 and performs the processes of steps S4 to S6 for another block. If the processes have been completed for all blocks (S7: YES), the state determination unit 11c stores state information, which associates the arrangement of the blocks detected from the photographed image with the state of each block, in the state information storage unit 12b in association with the photographed image stored in step S2 (step S8). The information transmission processing unit 11d of the processing unit 11 transmits the state information, which associates the arrangement and state of the blocks, to the terminal device 3 that is the transmission source of the photographed image acquired in step S1 (step S9), and ends the process.

[0053] <Summary> In the information processing system according to the present embodiment having the above configuration, the information processing device 1 acquires a photographed image of the bottom surface of the combustion chamber from the terminal device 3, detects components such as blocks or grates that make up the bottom surface of the combustion chamber from the acquired photographed image, determines the state of each detected component, and transmits (outputs) state information that associates the arrangement of multiple components with the state of each component to the terminal device 3. As a result, the information processing system according to the present embodiment is expected to easily and accurately determine the state of multiple components that make up the inner surface of the combustion chamber of an incinerator.

[0054] Furthermore, in the information processing system according to this embodiment, the information processing device 1 extracts an image area in which the detected component is captured from the captured image, inputs the extracted image area to a learning model, and obtains information output by the learning model to determine the state of the component. The learning model is a learning model that has been machine-learned in advance to receive an image in which the component is captured as an input and output information regarding the state of the component. As a result, the information processing system according to this embodiment is expected to easily and accurately determine the state of the component on the inner surface of the combustion chamber using the learning model that has been machine-learned in advance.

[0055] In addition, the information processing system according to the present embodiment handles blocks that constitute the walls of the combustion chamber as components that constitute the inner surface of the combustion chamber of the incinerator. This allows the information processing system according to the present embodiment to provide the user with information on the state of the blocks that constitute the walls of the combustion chamber of the incinerator, and is expected to encourage the user to repair or replace any abnormal blocks.

[0056] Furthermore, in the information processing system according to this embodiment, the terminal device 3 displays (outputs) status information in which the statuses of the multiple components are arranged in a matrix form according to the arrangement of the components on the display unit 34. As a result, the information processing system according to this embodiment can present the status of each component to the user in association with the actual arrangement of the combustion chamber inner surface, and is expected to enable the user to recognize the status of the combustion chamber inner surface.

[0057] In this embodiment, the information processing device 1 and the terminal device 3 cooperate to perform the processing, but this is not limited to this. The terminal device 3 may perform the processing of detecting the components and the processing of determining the state of the components, which are performed by the information processing device 1 in this embodiment. In this case, the information processing device 1 does not need to be included in the information processing system, and the terminal device 3 does not need to have a communication function. Furthermore, the method of displaying information on the state of the components determined by the information processing device 1 on the terminal device 3 is not limited to the display method shown in this embodiment, and various display methods may be adopted. Furthermore, the components to be subjected to the state determination are not limited to the blocks constituting the inner surface of the combustion chamber, but may be any objects other than blocks.

[0058] <Embodiment 2> The information processing system according to the second embodiment is a system for determining the condition of the fire grates that constitute the bottom surface of the combustion chamber of an incinerator. A stoker-type incinerator has multiple fire grates arranged in a stepped manner on the bottom surface of the combustion chamber, and these stepped fire grates can be moved back and forth to stably burn garbage, etc. For example, during regular inspections of the incinerator, the wear state of these multiple fire grates is inspected, and grates with a high degree of wear are replaced.

[0059] In the information processing system according to the embodiment, a user such as an operator or manager photographs the bottom surface of the combustion chamber using the terminal device 3 during regular inspection or the like, and transmits the photographed image from the terminal device 3 to the information processing device 1. The information processing device 1 according to the second embodiment detects the multiple grates that make up the bottom surface of the combustion chamber from the photographed image acquired from the terminal device 3, and determines the state of each detected grate. The information processing device 1 transmits status information to the terminal device 3 that associates the arrangement of the multiple grates detected from the photographed image with the state of each grate. The terminal device 3 receives the status information from the information processing device 1, and displays information related to the determination results of the states of the multiple grates on the display unit 34.

[0060] FIG. 8 is a schematic diagram showing an example of the detection of the grate. The upper part of FIG. 8 shows an example of a photographed image of the bottom surface of the combustion chamber photographed by the terminal device 3. The information processing device 1 acquires data of the photographed image of the bottom surface of the combustion chamber from the terminal device 3, and stores the acquired photographed image data in the state information storage unit 12b together with information such as the date and time of the photographed image and the identification information of the photographer. The information processing device 1 also performs a process of detecting a plurality of grates constituting the bottom surface of the combustion chamber photographed in the photographed image based on the photographed image data acquired from the terminal device 3. The lower part of FIG. 8 shows an example of the detection result of the grate for the photographed image in the upper part. In this embodiment, the bottom surface of the combustion chamber of the incinerator is composed of a square columnar grate arranged in a row, and in the photographed image of the bottom surface of the combustion chamber, a rectangular surface is photographed in a manner in which a plurality of rectangular surfaces are arranged in a matrix. The information processing device 1 extracts a boundary line separating a plurality of grates arranged in a matrix by performing image processing such as edge extraction on the photographed image, and detects a rectangular area surrounded by the boundary line as one grate.

[0061] The information processing device 1 according to the second embodiment performs a process of determining whether each grate detected from a captured image is in a normal state or in a worn state (abnormal state). The information processing device 1 receives an image of a grate as an input, and determines the state of a plurality of grates detected from the captured image using a learning model that has been machine-learned in advance to output information indicating the state of the grate captured in the image. Note that in the second embodiment, the state of the grate is in two states, a normal state and an abnormal state (worn state), but this is not limited thereto, and the state of the grate may be three or more.

[0062] The information processing device 1 extracts an image area in which each grate is captured from the captured image, inputs the extracted image area to a learning model, and obtains information on the state of the grate output by the learning model in response to this input, thereby determining the state of the grate. The information processing device 1 repeats the same process for multiple grates detected from the captured image, and determines the state of all grates detected from the captured image. After completing the state determination for all grates, the information processing device 1 generates state information that associates the arrangement of multiple grates detected from the captured image with the state of each grate, stores the generated state information in the state information storage unit 12b, and transmits it to the terminal device 3. The terminal device 3 can provide the user with information on the state of each grate on the bottom surface of the combustion chamber by performing a display based on the state information received from the information processing device 1.

[0063] The information processing system according to the second embodiment configured as above handles the fire grate that constitutes the bottom surface of the combustion chamber as a component that constitutes the inner surface of the combustion chamber of the incinerator. As a result, the information processing system according to the second embodiment is expected to provide the user with information on the state of the fire grate that constitutes the bottom surface of the combustion chamber of the incinerator, and to encourage the user to repair or replace the fire grate that has an abnormality.

[0064] Other configurations of the information processing system according to the second embodiment are similar to those of the information processing system according to the first embodiment, so the same reference numerals are used for the same parts and detailed description will be omitted.

[0065] <Embodiment 3> The information processing system according to the third embodiment photographs the interior of the combustion chamber of the incinerator with the camera 36 of the terminal device 3, measures the three-dimensional shape using the distance measuring sensor 37, and judges the state of the components such as blocks or grates that constitute the interior of the combustion chamber based on the photographed image and the three-dimensional shape data. Note that in the third embodiment, the components are described as being grates, but the components to be subjected to the state judgment are not limited to grates and may be blocks or other objects.

[0066] In the information processing system according to the third embodiment, the user photographs the bottom surface of the combustion chamber of the incinerator using the terminal device 3 at an appropriate timing, such as a regular inspection of the incinerator, and simultaneously measures the three-dimensional shape of the bottom surface of the combustion chamber using the distance measuring sensor 37. In this embodiment, the three-dimensional shape data of the bottom surface of the combustion chamber obtained by the measurement of the terminal device 3 is calculated by calculating the coordinates (X, Y, Z) of the object in the three-dimensional space based on the measurement result of the distance from the terminal device 3 to the object measured by the distance measuring sensor 37. For each pixel of the image obtained by the shooting of the camera 36, ​​the terminal device 3 associates information on the color (R, G, B) of the object captured in this pixel with the three-dimensional coordinates (X, Y, Z) of this object, and transmits the photographed image and three-dimensional shape data of the bottom surface of the combustion chamber to the information processing device 1. However, the terminal device 3 may transmit information on the distance (or depth, etc.) measured by the distance measuring sensor as the three-dimensional shape data, instead of the three-dimensional coordinates (X, Y, Z).

[0067] The information processing device 1 according to the third embodiment receives a photographed image and three-dimensional shape data of the bottom surface of the combustion chamber from the terminal device 3, and stores the received photographed image and three-dimensional shape data in the state information storage unit 12b together with information such as the photographed date and time and the identification information of the photographer. The information processing device 1 can detect a plurality of grates shown in the photographed image by extracting a boundary line separating the plurality of grates, for example, by performing image processing such as edge extraction based on the photographed image acquired from the terminal device 3. Note that when detecting a plurality of grates, the information processing device 1 may use the three-dimensional shape data instead of the photographed image, or may use both the photographed image and the three-dimensional shape data.

[0068] The information processing device 1 performs processing to predict the state of a plurality of grates captured in a captured image using a learning model that has been machine-learned in advance based on the captured image and three-dimensional shape data acquired from, for example, the terminal device 3. FIG. 9 is a schematic diagram showing an example of a configuration of a learning model used by the information processing device 1 according to the third embodiment. The learning model according to the third embodiment is generated in advance by machine learning as a classification model that accepts, for example, captured image data and three-dimensional shape data as inputs and classifies the grates captured in the captured image as either normal or abnormal (worn). The learning model is, for example, a learning model configured as a CNN or the like, and the information processing device 1 generates in advance a learning model that classifies the state of the grates by performing so-called supervised machine learning using learning data that associates, for example, an image of the grate and the three-dimensional shape data of the grate with the state of the grate.

[0069] The information processing device 1 according to the third embodiment extracts an image area in which each grate is captured from the captured image, extracts three-dimensional shape data (hereinafter referred to as partial shape data) corresponding to each grate from the three-dimensional shape data, inputs the extracted image area and partial shape data to a learning model, obtains information on the state of the grate output by the learning model in response to this input, and determines the state of each grate. The information processing device 1 repeatedly performs the same process for multiple grates detected from the captured image, and determines the state of all grates detected from the captured image.

[0070] Furthermore, in the information processing system according to the third embodiment, the amount of thinning can be calculated as a numerical value indicating the degree of wear of each grate based on the three-dimensional shape data acquired from the terminal device 3. In this embodiment, the amount of thinning is the amount of reduction (change) in the thickness of the grate, and is an amount expressed in units such as meters, centimeters, or millimeters. In order to calculate the amount of thinning, the user photographs the bottom surface of the combustion chamber and measures the three-dimensional shape data using the terminal device 3, for example, before the incinerator is operated (i.e., when the grate is not worn), and transmits the obtained photographed image and three-dimensional shape data to the information processing device 1 as initial data. The information processing device 1 stores this initial data in the state information storage unit 12b, and calculates the amount of change (difference) between the three-dimensional shape data of the initial data and the three-dimensional shape data obtained during subsequent regular inspections, etc., to calculate the amount of thinning of each grate. For example, the information processing device 1 calculates the amount of thinning for each grate detected from a captured image at a point corresponding to each pixel in the image area where the grate is photographed, calculates statistical values ​​such as the average, maximum or median of multiple amounts of thinning corresponding to multiple pixels, and can treat the calculated statistical values ​​as the amount of thinning for that grate.

[0071] The information processing device 1, which has completed the state determination and calculation of the amount of metal loss for all grates detected from the photographed images, generates state information that associates the arrangement of the multiple grates that make up the bottom surface of the combustion chamber, the state of each grate, and the amount of metal loss for each grate. The information processing device 1 stores the generated state information in the state information storage unit 12b in association with the photographed images and the three-dimensional shape data, and transmits the generated state information to the terminal device 3 that transmitted the photographed images and the three-dimensional shape data. The terminal device 3 receives the state information transmitted from the information processing device 1 and performs display based on the received state information.

[0072] The terminal device 3 can display the arrangement of the multiple grates in association with the state of each grate, for example, in the same manner as the display example of the state information shown in Fig. 6. Furthermore, in the information processing system according to the third embodiment, the terminal device 3 displays the calculated value of the amount of thinning for each grate in multiple squares corresponding to the multiple grates. The terminal device 3 may also display the squares corresponding to the grates in different colors according to the amount of thinning, and in this case, the corresponding squares of grates determined to be abnormal may be highlighted by blinking or surrounding them with a thick frame.

[0073] 10 is a flowchart showing an example of a procedure of a state determination process performed by the information processing device 1 according to the embodiment 3. The data acquisition unit 11a of the processing unit 11 of the information processing device 1 according to the embodiment 3 acquires data of a photographed image of the bottom surface of the combustion chamber photographed by the camera 36 of the terminal device 3 and three-dimensional shape data of the bottom surface of the combustion chamber measured by the distance measurement sensor 37 (step S21). At this time, the data acquisition unit 11a also acquires information such as the photographing date and time and the identification information of the photographer from the terminal device 3, and stores the photographed image data and the three-dimensional shape data in the state information storage unit 12b in association with this information (step S22).

[0074] Next, the component detection unit 11b of the processing unit 11 performs a process to detect the multiple fire grates constituting the bottom surface of the combustion chamber captured in the captured image based on the data of the captured image acquired in step S21 (step S23). At this time, the component detection unit 11b performs image processing such as edge extraction on the captured image to extract the boundary lines separating the multiple fire grates, and detects a rectangular area surrounded by vertical and horizontal boundaries as one fire grate. Next, the state determination unit 11c of the processing unit 11 acquires initial shape data of the three-dimensional shape of the bottom surface of the combustion chamber measured before the incinerator is operated, which is stored in advance in the state information storage unit 12b (step S24).

[0075] Next, based on the detection result of step S23, the state determination unit 11c extracts an image area in which one grate is captured from the captured image for one of the multiple grates detected from the captured image, and extracts partial shape data corresponding to this grate from the three-dimensional shape data (step S25). The state determination unit 11c inputs the image area and partial shape data of the grate extracted in step S25 to a learning model that has been machine-learned in advance (step S26). The state determination unit 11c acquires information output by the learning model in response to the input of the image area and partial shape data in step S26 (step S27). The state determination unit 11c can determine the state of the grate based on the information output by the learning model. The state determination unit 11c also calculates the amount of wall thinning of the grate by calculating the difference between the part related to the target grate in the initial data read out in step S24 and the partial shape data extracted in step S25 (step S28).

[0076] The state determination unit 11c determines whether the processing of steps S25 to S28 has been completed for all grates detected in step S23 (step S29). If the processing has not been completed for all grates (S29: NO), the state determination unit 11c returns the processing to step S25 and performs the processing of steps S25 to S28 for another grate. If the processing has been completed for all grates (S29: YES), the state determination unit 11c stores state information, which associates the arrangement of the multiple grates detected from the photographed image, the state of each grate, and the amount of thinning of each grate, in the state information storage unit 12b in association with the photographed image and the three-dimensional shape data stored in step S22 (step S30). The information transmission processing unit 11d of the processing unit 11 transmits state information, which associates the arrangement, state, and amount of thinning of the grates, to the terminal device 3 that transmitted the photographed image and the three-dimensional shape data acquired in step S21 (step S31), and ends the processing.

[0077] In the information processing system according to the third embodiment configured as above, the information processing device 1 acquires from the terminal device 3 the image captured by the camera 36 and the three-dimensional shape data of the inner surface of the combustion chamber based on the measurement of the distance measuring sensor 37, and determines the state of the components that make up the inner surface of the combustion chamber based on the captured image and the three-dimensional shape data. As a result, the information processing system according to the third embodiment is expected to easily and more accurately determine the state of the multiple components that make up the inner surface of the combustion chamber of an incinerator.

[0078] Moreover, in the information processing system according to the third embodiment, the information processing device 1 extracts an image area in which the components are captured from the captured image, extracts partial shape data corresponding to the components from the 3D shape data, inputs the extracted image area and partial shape data to a learning model for which machine learning has been performed in advance, and obtains information output by the learning model, thereby determining the state of each component captured in the captured image. As a result, the information processing system according to the third embodiment is expected to easily and more accurately determine the state of the components on the inner surface of the combustion chamber by using a learning model for which machine learning has been performed in advance.

[0079] In the information processing system according to the third embodiment, the information processing device 1 stores the three-dimensional shape data acquired from the terminal device 3 in the state information storage unit 12b. The information processing device 1 calculates the amount of thinning (amount of change in shape data) of each component constituting the inner surface of the combustion chamber based on the three-dimensional shape data (initial shape data) measured at a first time point, such as before the incinerator is put into operation, and the three-dimensional shape data measured at a second time point, such as during a regular inspection thereafter, and transmits (outputs) state information in which the arrangement of the multiple components, the state of each component, and the amount of thinning of each component are associated with each other to the terminal device 3. This allows the information processing system according to the third embodiment to provide the user with information on the amount of thinning in addition to the state of each component.

[0080] Other configurations of the information processing system according to the third embodiment are similar to those of the information processing systems according to the first and second embodiments, so the same reference numerals are used for the same parts and detailed description will be omitted.

[0081] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present disclosure is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims.

[0082] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this. The claims may be written in a format in which a multiple claim cites at least one other multiple claim (multi-multi claim). [Explanation of symbols]

[0083] 1. Information processing equipment (computer) 3 Terminal Equipment 11 Processing section 11a Data Acquisition Section 11b Component detection unit 11c Status determination section 11d Information transmission processing section 12 Storage section 12a Programs (computer programs) 12b Status information storage unit 13. Communications Department 31 Processing section 31a Data transmission processing unit 31b Display processing unit 32 Storage section 32a Program 33 Communications Department 34 Display section 35 Control section 36 Camera 37 Distance Sensor 99 Recording media N Network

Claims

1. On the computer, Obtaining photographic images of the combustion chamber interior, Detecting a plurality of components constituting the combustion chamber inner surface from the photographed image; The state of each detected component is determined. Outputting state information in which the arrangement of the plurality of components is associated with the state of each component. A computer program that executes a process.

2. Extracting an image area in which the detected component is captured from the captured image; The extracted image area is input to a learning model that has been machine-learned to receive an image in which the components are captured as an input and output the state of the components, and the state output by the learning model is obtained, thereby determining the state of each component captured in the captured image.

2. The computer program product of claim 1.

3. acquiring shape data of the inner surface of the combustion chamber based on measurements by a distance measuring sensor; determining a state of the component based on the captured image and the shape data; 2. The computer program product of claim 1.

4. Extracting an image area in which the detected component is captured from the captured image; Extracting partial shape data corresponding to the component from the shape data; inputting the extracted image area and the partial shape data into a learning model that has been machine-learned to receive an image in which the components are captured and shape data of the components as inputs and output the states of the components, and acquiring the states output by the learning model, thereby determining the states of each of the components captured in the captured image; 4. A computer program according to claim 3.

5. The acquired shape data is stored in a storage unit; Calculating an amount of change in the shape data of each of the components based on the shape data measured at a first time point and the shape data measured at a second time point after the first time point; outputting state information in which the arrangement of the plurality of components, the state of each component, and the amount of change of each component are associated with each other; 4. A computer program according to claim 3.

6. The component is a block that constitutes the wall surface of the combustion chamber.

2. The computer program product of claim 1.

7. The component is a fire grate that constitutes the bottom surface of the combustion chamber.

2. The computer program product of claim 1.

8. outputting the state information in which the states of the components are arranged in a matrix according to an arrangement of the plurality of components; 2. The computer program product of claim 1.

9. An information processing device, Obtaining photographic images of the combustion chamber interior, Detecting a plurality of components constituting the combustion chamber inner surface from the photographed image; The state of each detected component is determined. outputting state information in which the arrangement of the plurality of components is associated with a state of each of the components; Information processing methods.

10. A processing unit is provided, The processing unit, Obtaining photographic images of the combustion chamber interior, Detecting a plurality of components constituting the combustion chamber inner surface from the photographed image; The state of each detected component is determined. outputting state information in which the arrangement of the plurality of components is associated with a state of each of the components; Information processing device.

Images