Stoker type processing device, method for processing object to be processed, and processing program

Abstract

To simply the process of acquiring the stack state of the objects to be processed in a stoker.SOLUTION: A stoker type processing device comprises: a stoker for supporting the object to be processed and transporting the object to be processed in a transport direction that intersects the vertical direction; an imaging device for capturing an image of a field of view in which the plurality of sections of the stoker is included; a region evaluation unit for identifying, on the basis of a two-dimensional field-of-view image obtained through image capturing by the imaging device, a target region of the field-of-view image where the object to be processed exists; a distance distribution evaluation unit for acquiring distance distribution information indicating the distance from the imaging device to the object in the field of view; and a stack state evaluation unit for acquiring the relative relationship of the stack state of the objects to be processed in the plurality of sections on the basis of stack state information obtained by combining the distance distribution information and the result of target region identification.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a stoker-type treatment device, a treatment method for an object to be treated, and a treatment program. [Background technology]

[0002] Patent document 1 discloses a stoker-type incinerator equipped with a thermal image capturing unit that captures a thermal image of garbage in a target area, which is at least a portion of the area on the grate section, and a calculation unit that acquires garbage height information indicating the distribution of garbage height over the target area based on the thermal image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-187228 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a stoker-type treatment device, a treatment method for a treatment object, and a treatment program that are useful for simplifying the process of acquiring the accumulation status of a treatment object on a stoker. [Means for solving the problem]

[0005] [1] A stoker-type treatment apparatus comprising: a stoker that supports an object to be treated and transports the object in a transport direction that intersects the vertical direction; an imaging device that images a field of view that includes multiple compartments on the stoker; an area evaluation unit that identifies a target area in the field of view image in which the object to be treated exists based on a two-dimensional image within the field of view obtained by imaging with the imaging device; a distance distribution evaluation unit that obtains distance distribution information that indicates the distance from the imaging device to an object within the field of view; and a deposition situation evaluation unit that obtains the relative relationship of the deposition situations of the object to be treated in the multiple compartments based on deposition situation information obtained by combining the distance distribution information with the result of identifying the target area.

[0006] [2] The stoker-type treatment device described in [1] above, wherein the multiple compartments are aligned in the conveying direction.

[0007] [3] The stalker-type processing device described in [1] or [2] above, wherein the imaging device focuses mid-infrared light incident from the field of view to generate an image within the field of view.

[0008] [4] The stalker-type processing device described in any one of [1] to [3] above, wherein the distance distribution evaluation unit acquires the distance distribution information by inputting the image within the field of view to a distance estimation model constructed by machine learning so as to output information representing the distribution of distances to objects in a two-dimensional image in response to an input of the two-dimensional image.

[0009] [5] A stoker-type treatment device described in any one of [1] to [4] above, wherein the deposition situation evaluation unit obtains the relative relationship by inputting the deposition situation information into a judgment model constructed by machine learning so as to output information indicating the relative relationship of the deposition situations of the treatment object in the multiple sections in response to input information corresponding to the deposition situation information.

[0010] [6] A stoker-type treatment device as described in any one of [1] to [4] above, wherein the distance distribution information includes information indicating the distance for each pixel in the image within the field of view, and the deposition status evaluation unit calculates, for each pixel in the deposition status information, three-dimensional coordinates consisting of the two-dimensional coordinates of the pixel and the distance, and acquires the relative relationship based on area information indicating regions corresponding to each of the multiple sections on a three-dimensional graph with the two-dimensional coordinates and the distance as variables, and the calculation results of the three-dimensional coordinates for each pixel in the deposition status information.

[0011] [7] The stalker-type processing device described in any one of [1] to [6] above, wherein the area evaluation unit identifies the target area by inputting the image within the field of view to an area identification model constructed by semantic segmentation so as to output, in response to an input of a two-dimensional image, information indicating the result of classifying each pixel in the two-dimensional image into an area where the processing object exists and an area where the processing object does not exist.

[0012] [8] The stalker-type processing device described in any one of [1] to [6] above, wherein the area evaluation unit identifies the target area by performing a process of comparing the standard deviation of pixel values ​​in a divided area of ​​the image within the field of view with a threshold value while sliding the coordinates of a representative point of the divided area.

[0013] [9] The stalker-type processing device according to any one of [1] to [6] above, wherein the area evaluation unit identifies the target area by executing a process of detecting edges in the image within the field of view.

[0014]

[10] The stalker-type processing device described in any one of [1] to [6] above, wherein the area evaluation unit acquires a plurality of in-field images including the in-field image, the plurality of in-field images being obtained by continuing imaging by the imaging device for a predetermined period of time, and identifies the target area by performing a process of comparing, for each pixel in the plurality of in-field images, a standard deviation in the change in pixel value over time with a threshold value.

[0015]

[11] A stoker type treatment device described in any one of [1] to

[10] above, further comprising a control unit that adjusts the treatment of the treatment object on the stoker so as to reduce differences in the accumulation conditions of the treatment object among the multiple sections based on the relative relationship acquired by the accumulation condition evaluation unit.

[0016]

[12] A method for processing an object to be processed, comprising: a transporting step of transporting the object to be processed in a transport direction intersecting a vertical direction by a stoker supporting the object to be processed; an imaging step of imaging a field of view including a plurality of compartments on the stoker by an imaging device; an area evaluation step of identifying a target area in the field of view image in which the object to be processed exists based on a two-dimensional image within the field of view obtained by imaging with the imaging device; a distance distribution evaluation step of acquiring distance distribution information indicating the distance from the imaging device to an object within the field of view; and a deposition situation evaluation step of acquiring a relative relationship of deposition situations of the object to be processed in the plurality of compartments based on deposition situation information obtained by combining the distance distribution information and the result of identifying the target area.

[0017]

[13] A processing program for causing a computer to execute the following steps: an image acquisition step for acquiring a two-dimensional image within a field of view obtained by capturing an image of a field of view including multiple compartments on a stoker that supports an object to be processed and transports the object to be processed in a transport direction that intersects the vertical direction using an imaging device; an area evaluation step for identifying a target area within the field of view image in which the object to be processed exists based on the image within the field of view; a distance distribution evaluation step for acquiring distance distribution information indicating the distance from the imaging device to an object within the field of view; and a deposition situation evaluation step for acquiring the relative relationship of the deposition situations of the object to be processed in the multiple compartments based on deposition situation information obtained by combining the distance distribution information with the result of identifying the target area. Effect of the Invention

[0018] According to the present disclosure, there is provided a stoker-type treatment device, a treatment method for a treatment object, and a treatment program that are useful for simplifying the process of acquiring the accumulation status of the treatment object on a stoker. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a schematic diagram showing an example of a stoker-type treatment device. [Diagram 2] FIG. 2 is a plan view that illustrates an example of a stoker. [Diagram 3] FIG. 3 is a schematic diagram showing an example of an image obtained by the imaging device. [Figure 4] FIG. 4 is a block diagram illustrating an example of a functional configuration of the control device. [Diagram 5] FIG. 5 is a schematic diagram showing an example of a process for acquiring a distance distribution. [Figure 6] FIG. 6 is a schematic diagram showing an example of a process of identifying a target region. [Figure 7] FIG. 7 is a diagram showing an example of the accumulation status information. [Figure 8] FIG. 8 is a schematic diagram showing an example of a process for determining the relative relationship of the accumulation state. [Figure 9] FIG. 9 is a schematic diagram showing an example of learning data when constructing a determination model. [Figure 10] FIG. 10 is a block diagram illustrating an example of a hardware configuration of the control device. [Figure 11] FIG. 11 is a flowchart showing an example of a series of control processes executed by the control device. [Figure 12] FIG. 12 is a diagram showing an example of a process of identifying a target region. [Figure 13] FIG. 13 is a diagram showing an example of a process of identifying a target region. [Figure 14] Fig. 14(a) is a schematic diagram illustrating the relationship between pixels of an image within a field of view and three-dimensional coordinates, and Fig. 14(b) is a diagram illustrating three-dimensional plot data obtained from deposition status information. [Figure 15] FIG. 15 is a schematic diagram showing an example of information indicating a plurality of sections on a three-dimensional graph. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment will be described with reference to the drawings. In the description, the same elements or elements having the same functions are denoted by the same reference numerals, and duplicated description will be omitted.

[0021] Fig. 1 shows a schematic diagram of a stoker type treatment apparatus according to one embodiment. The stoker type treatment apparatus 1 shown in Fig. 1 is an apparatus that heat-treats an object to be treated while transporting the object in a transport direction that intersects with the vertical direction. In Fig. 1, the transport direction is indicated by "D1", and the transport direction D1 is, for example, a horizontal direction. The object to be treated by the stoker type treatment apparatus 1 is, for example, waste such as industrial waste or general waste.

[0022] When the object to be treated is waste, the stoker type treatment device 1 combusts (thermally treats) the waste supplied by the dust feeder 14 and sends it to a main ash chute. The stoker type treatment device 1 is also called a stoker type incinerator. Note that the object to be treated is not limited to waste. The object to be treated may be biomass. The stoker type treatment device 1 includes a stoker furnace 10, an imaging device 90, and a control device 100.

[0023] The stoker furnace 10 accommodates the object to be treated and transports the object to be treated in a transport direction D1 that intersects (e.g., perpendicular to) the vertical direction. The stoker furnace 10 has, for example, a furnace body 11, a stoker 30, a drive device 33, and a blower section 40. The furnace body 11 accommodates the object to be treated. The furnace body 11 forms a storage space that extends along the transport direction D1. The furnace body 11 has a receiving section 12 and a sending section 13 at both ends of the furnace body 11. The receiving section 12 accepts the object to be treated into the space within the furnace body 11. The sending section 13 sends the object to be treated out of the space within the furnace body 11.

[0024] When the material to be treated is waste, the receiving section 12 receives the waste supplied by a dust feeder 14. The sending section 13 sends the waste to a main ash chute provided below the furnace body 11.

[0025] The stoker 30 is provided at the bottom of the furnace body 11, supports the object to be treated, and transports the object to be treated in the transport direction D1. The stoker 30 has, for example, a plurality of fixed grates 31 and a plurality of movable grates 32. The plurality of fixed grates 31 are aligned along the transport direction D1. The plurality of movable grates 32 are provided so as to correspond to the plurality of fixed grates 31, respectively. Each of the plurality of movable grates 32 is provided, for example, on the corresponding fixed grate 31.

[0026] The driving device 33 reciprocates each of the multiple movable grates 32 along the conveying direction D1. The driving device 33 reciprocates each of the multiple movable grates 32 along the conveying direction D1, for example, by an electric motor or a hydraulic cylinder. The driving device 33 is configured to be able to change the grate speed of the multiple movable grates 32 individually.

[0027] The grate speed is the transport speed of the material to be treated by the reciprocating movement of the movable grate 32. To change the grate speed, the driving device 33 may change the displacement speed of the movable grate 32 in one reciprocating movement, may change the displacement stroke of the movable grate 32 in one reciprocating movement, or may change the number of reciprocating movements of the movable grate 32 per unit time. To change the grate speed, the driving device 33 may change two or more of the displacement speed, the displacement stroke, and the number of reciprocating movements.

[0028] The heights of the multiple fixed grates 31 may decrease in a stepped manner toward the conveying direction D1 (downstream in the conveying direction D1). In this case, the object to be treated moves from one fixed grate 31 to the next fixed grate 31 by the amount of the step between the fixed grates 31. In this way, conveying the object to be treated in the conveying direction D1 includes displacing the object to be treated in a direction different from the conveying direction (for example, downward) while displacing the object to be treated in the conveying direction D1.

[0029] In this disclosure, the terms "upstream" and "downstream" are used based on the direction in which the material to be treated is transported. That is, in the stoker-type treatment device 1, the material to be treated is transported from upstream to downstream. In addition, the direction from upstream to downstream is referred to as "forward," and the direction from downstream to upstream is referred to as "rearward." The terms "right" and "left" are used based on the view from upstream to downstream (viewing forward).

[0030] Each of the multiple movable grates 32 may be divided into multiple zones 34 arranged in the width direction D2 of the furnace body 11 (see FIG. 2). The width direction D2 is a direction perpendicular to both the vertical direction and the conveying direction D1. Each of the multiple movable grates 32 is divided into two zones 34, for example, a zone 34 located on the right side and a zone 34 located on the left side. The drive device 33 may be configured to be able to change the grate speed of the multiple zones 34 individually.

[0031] The blower 40 sends gas for heat treatment (e.g., for combustion) from below the stoker 30 through the multiple fixed grates 31 and the movable grate 32 to the object to be treated. The gas for heat treatment is, for example, an oxygen-containing gas such as air. The gas for heat treatment may be at room temperature or may be preheated. The blower 40 may be configured to be able to individually change the amount of air blown to the multiple blowing areas 43 on the stoker 30. The multiple blowing areas 43 may correspond to the multiple fixed grates 31, respectively.

[0032] The air blowing section 40 has, for example, an air blowing source 41 and a plurality of valves 42. The air blowing source 41 pressure-feeds gas for heat treatment to a plurality of air blowing areas 43, for example by a blower or the like. The plurality of valves 42 adjust the flow rate of gas from the air blowing source 41 to the plurality of air blowing areas 43, respectively. Each of the plurality of air blowing areas 43 may be divided into a plurality of air blowing zones 44 corresponding to the plurality of zones 34, respectively (see FIG. 2), and the air blowing section 40 may be configured to be able to change the amount of air blown to each of the plurality of air blowing zones 44 individually.

[0033] The imaging device 90 captures an image within a predetermined field of view FV on the stoker 30. The imaging device 90 is directed toward a predetermined field of view FV on the stoker 30 from the front in the conveying direction D1. For example, the imaging device 90 is located forward of the fixed grate 31 located at the most downstream in the conveying direction D1 and directed toward the rear. The imaging device 90 may be disposed such that the field of view (direction toward the center of the field of view FV) is directed from obliquely above toward the upper surface of the stoker 30. For example, the imaging device 90 is disposed at a position higher than the stoker 30 and directed obliquely downward. The imaging device 90 may be provided such that each of the multiple fixed grates 31 is at least partially within the field of view FV.

[0034] The imaging device 90 has a predetermined viewpoint VP, and generates a two-dimensional image (image data) as information for acquiring a distance distribution from the viewpoint VP to an object (substance) in the field of view FV. Hereinafter, the two-dimensional image generated by the imaging device 90 is referred to as an "in-field image VI". The imaging device 90 generates the in-field image VI based on light from the field of view FV. The imaging device 90 may be configured to generate the in-field image VI by, for example, forming an image of mid-infrared light incident from the field of view FV. Mid-infrared light is an electromagnetic wave with a wavelength of approximately 2.5 μm to approximately 4 μm. With mid-infrared light, it is possible to acquire information that is not easily influenced by flames, water vapor, carbon dioxide gas, and the like generated by the combustion of the processing object.

[0035] 3 is a schematic diagram showing an example of a field-of-view image VI acquired by imaging by the imaging device 90. As shown in FIG. 3, the field-of-view FV of the imaging device 90 includes a plurality of sections SC corresponding to the plurality of deposition areas on the stoker 30, respectively. The plurality of deposition areas are areas in which the material to be treated is deposited. For example, the plurality of deposition areas are areas obtained by subdividing the upper surface of the stoker 30. In one example, the plurality of deposition areas correspond to the plurality of fixed grates 31 described above.

[0036] One of the multiple sections SC corresponds to one of the multiple fixed grates 31. One section SC may be divided into two zones 34 (two ventilation zones 44) arranged side by side. In the example shown in FIG. 3, the multiple sections SC are arranged in the conveying direction D1. Each of the multiple sections SC can be considered as an area divided in the conveying direction D1 based on the fixed grate 31. For example, "section 1", "section 2", "section 3", "section 4", and "section 5" are arranged in order from the upstream side. Each of the multiple sections SC can also be referred to as a combustion zone.

[0037] The control device 100 controls the treatment of the object to be treated on the stoker 30. Controlling the treatment of the object to be treated on the stoker 30 includes at least controlling the drive device 33 or the blower unit 40. In order to control the treatment of the object to be treated, the control device 100 may control either the drive device 33 or the blower unit 40, or may control both the drive device 33 and the blower unit 40.

[0038] In order to improve the efficiency of the treatment of the treatment object (for example, heat treatment including incineration) in the stoker furnace 10, it is necessary to appropriately distribute the degree of accumulation of the treatment object on the stoker 30. For example, if the amount of the treatment object accumulated at a certain point on the stoker 30 is greater than that at other points, the treatment at that point may not proceed sufficiently. Therefore, the control device 100 evaluates the relative relationship of the accumulation status of the treatment object in multiple sections on the stoker 30, and controls the drive device 33, etc. to adjust the accumulation status of the treatment object based on the evaluation result. The configuration of the control device 100 will be described in detail below.

[0039] Fig. 4 illustrates an example of the functional configuration of the control device 100. As shown in Fig. 4, the control device 100 has, for example, a distance distribution evaluation unit 112, a distance distribution storage unit 116, an area evaluation unit 118, a deposition status evaluation unit 120, and a control unit 122 as functional components (hereinafter referred to as "functional blocks"). The processing executed by these functional blocks corresponds to the processing executed by the control device 100. Hereinafter, the processing executed by the control device 100 will be referred to as "control processing" in order to distinguish it from the physical processing of the processing target on the stoker 30.

[0040] The distance distribution evaluation unit 112 acquires distance distribution information DI indicating distances from the imaging device 90 to objects within the field of view FV. The distance distribution evaluation unit 112 stores the acquired distance distribution information DI in the distance distribution storage unit 116. The distance distribution evaluation unit 112 acquires the distance distribution information DI from the in-field of view image VI using, for example, a distance estimation model M1.

[0041] 5 shows a schematic diagram of a process of obtaining distance distribution information DI from a visual field image VI using a distance estimation model M1. The distance estimation model M1 is a trained model constructed by machine learning so as to output information representing a distribution of distances to objects in a two-dimensional image in response to an input of a two-dimensional image. The distance distribution evaluation unit 112 may obtain, as the distance distribution information DI, information output from the distance estimation model M1 when the visual field image VI is input to the distance estimation model M1.

[0042] In one example, the distance distribution information DI is image information having a pixel value for each pixel in the image VI within the field of view, and includes information (pixel value) indicating the distance for each pixel in the image VI within the field of view. In the distance distribution information DI, the magnitude of the pixel value represents the magnitude of the distance. For example, the distance distribution information DI is grayscale information, and the darker the color (closer to black), the shorter the distance, and the lighter the color (closer to white), the longer the distance.

[0043] The distance estimation model M1 is constructed in advance by machine learning based on a plurality of learning data sets so as to output information representing a distribution of distances to objects in a two-dimensional image in response to an input of a two-dimensional image. Each of the plurality of learning data sets includes, for example, data relating to the two-dimensional image and actual measurement data of a distance distribution indicating a distance to an object for each pixel in the two-dimensional image. The distance estimation model M1 may be constructed based on a plurality of learning data sets acquired in a space different from the stoker 30 (a space outside the stoker furnace 10). Specific examples of learning algorithms used to construct the distance estimation model M1 include ViT (Vision Transformer) or a neural network.

[0044] The area evaluation unit 118 identifies an area (hereinafter referred to as a "target area Z1") in which a processing target exists in the field of view image VI based on a two-dimensional field of view image VI obtained by imaging by the imaging device 90. The area evaluation unit 118 identifies the target area Z1 in the field of view image VI by using, for example, an area identification model M2.

[0045] 6 shows a schematic diagram of a process of identifying a target region Z1 in a visual field image VI using a region identification model M2. The region identification model M2 is a trained model constructed by semantic segmentation so as to output information indicating the result of classifying each pixel in a two-dimensional image into a region where a processing object exists and a region where a processing object does not exist. Semantic segmentation is a type of machine learning algorithm.

[0046] The area identification model M2 outputs a classification result for each pixel in the image VI within the field of view, whether the pixel is an area where the processing target exists or an area where the processing target does not exist. As a result, in the data output from the area identification model M2, the area (whole area) on the image corresponding to the image VI within the field of view is divided into a target area Z1 and an area other than the target area Z1 (hereinafter referred to as a "non-target area Z2"). In one example, the area evaluation unit 118 acquires a data set indicating the coordinates of each of all pixels divided into the target area Z1 as information indicating the identification result of the target area Z1.

[0047] The region identification model M2 is constructed in advance by machine learning using semantic segmentation based on a plurality of learning data sets so as to output information indicating the result of classifying each pixel in the two-dimensional image into a region where the processing object exists and a region where the processing object does not exist in response to the input of a two-dimensional image. The plurality of learning data sets include, for example, data related to the two-dimensional image and correct answer data indicating whether or not the processing object exists for each pixel in the two-dimensional image. The correct answer data regarding whether or not the processing object exists may be created by converting the result of a judgment made by a person such as an operator of the stoker type processing device 1 into data. The region identification model M2 may be constructed based on a plurality of learning data sets acquired in a space different from the stoker 30 (a space outside the stoker furnace 10).

[0048] The accumulation status evaluation unit 120 acquires a relative relationship of accumulation statuses of the processing target in the multiple sections SC based on accumulation status information DI1 obtained by combining the distance distribution information DI and the result of identifying the target area Z1. The accumulation status evaluation unit 120 first generates accumulation status information DI1 by combining (based on) the distance distribution information DI stored in the distance distribution storage unit 116 and information indicating the result of identifying the target area Z1.

[0049] 7 shows a schematic process of generating deposition status information DI1 by combining distance distribution information DI and the result of identifying the target area Z1. The deposition status information DI1 is image information obtained by extracting the target area Z1 from the distance distribution information DI, which is image information, or image information obtained by masking with a non-target area Z2, which is an area other than the target area Z1. In the deposition status information DI1, for example, the coordinates of all pixels in a part of the distance distribution information DI divided into the target area Z1 correspond to a pixel value indicating the magnitude of the distance (depth). The deposition status information DI1 does not need to include information (coordinates and pixel values) regarding the part of the distance distribution information DI divided into the non-target area Z2.

[0050] The accumulation status information DI1 indicates at what position (coordinate) the processing target is present in the visual field FV (visual field image VI) and, for each coordinate at which the processing target is present, how far away the processing target is from that coordinate. In this way, the accumulation status information DI1 is information that indicates the accumulation status of the processing target on the stoker 30. Therefore, the accumulation status information DI1 includes information that indicates the relative relationship of the accumulation status of the processing target in multiple sections.

[0051] The information indicating the relative relationship of the accumulation state is, for example, information indicating in which section the processing object is most accumulated. The information indicating the relative relationship of the accumulation state may include information specifying one or more sections in which the processing object is most accumulated, and a numerical value indicating how much the processing object is accumulated in the one or more sections compared to other sections. Alternatively, the information indicating the relative relationship of the accumulation state may be the relative amount of the processing object for each section. For example, when sections 1 to 5 are set, the relative amount of the processing object for each section can be expressed as (section 1, section 2, section 3, section 4, section 5) = (10%, 10%, 10%, 60%, 10%).

[0052] After generating the distance distribution information DI, the accumulation situation evaluation unit 120 acquires the relative relationship of the accumulation situations of the processing objects in the multiple sections SC based on the distance distribution information DI. The accumulation situation evaluation unit 120 acquires the relative relationship of the accumulation situations of the processing objects in the multiple sections SC, for example, by using a determination model M3.

[0053] 8 shows a schematic diagram of a process of acquiring the relative relationship of the deposition status of the processing target in the multiple sections SC using the determination model M3. The determination model M3 is a trained model constructed by machine learning so as to output information indicating the relative relationship of the deposition status of the processing target in the multiple sections SC in response to input of input information corresponding to the deposition status information DI1. The input information corresponding to the deposition status information DI1 means information of the same type as the deposition status information DI1.

[0054] The input data to the judgment model M3 is deposition status information DI1 obtained from a combination of the distance distribution information DI and the result of identifying the target area Z1. The output data from the judgment model M3 (indicated as "OD" in FIG. 8) is information indicating the relative relationship of the deposition status of the processing object in the multiple sections SC. The output from the judgment model M3 need only be information indicating the relative relationship of the deposition status of the processing object in the multiple sections SC, and does not have to be image information. In FIG. 8 etc., the relative relationship of the deposition status is displayed as a diagram to clearly show the calculation contents of the judgment model M3.

[0055] The judgment model M3 is constructed in advance by machine learning based on a plurality of learning data sets so as to output the relative relationship of the objects to be processed in a plurality of sections SC in response to the input of the accumulation status information DI1. FIG. 9 illustrates a plurality of learning data sets used in constructing the judgment model M3. Each of the plurality of learning data sets includes, for example, the accumulation status information DI1 and ground truth data TD (performance data) indicating the relative relationship of the accumulation status of the objects to be processed when the accumulation status information DI1 is obtained. Specific examples of the learning algorithm used in constructing the judgment model M3 include ViT or a neural network.

[0056] When preparing a plurality of learning data sets, the correct answer data TD regarding the relative relationship of the deposition conditions of the objects to be processed may be created by digitizing the results of judgment made by a person such as the operator of the stoker-type processing device 1. The operator preparing the correct answer data TD may determine the correct answer data TD while viewing the deposition condition information DI1 displayed as an image and the field-of-view image VI that is the source of the deposition condition information DI1.

[0057] In an image acquired by the imaging device 90 (camera), the size of an object on the image changes depending on the distance (depth) to the object. That is, even if an object is actually the same size, if the distance from the camera is large, the size on the image will be small, and if the distance from the camera is small, the size on the image will be large. As a result, if the degree of accumulation of the processing objects for each section is simply evaluated on the image, there is a concern that the processing objects in the foreground will be overestimated and the processing objects in the background will be underestimated. Therefore, an operator or the like who prepares the correct answer data TD may determine the correct answer data TD taking into account the depth on the image (the distance from the imaging device 90).

[0058] As shown in Fig. 4, the control device 100 may have a model construction unit 124 and a model storage unit 126 as functional blocks. The model construction unit 124 constructs at least a part of the distance estimation model M1, the area identification model M2, and the judgment model M3 (e.g., each of all three trained models) by machine learning. The model construction unit 124 stores the constructed one or more trained models (e.g., the distance estimation model M1, the area identification model M2, and the judgment model M3) in the model storage unit 126. Each of the distance distribution evaluation unit 112, the area evaluation unit 118, and the accumulation status evaluation unit 120 may use the corresponding trained model stored in the model storage unit 126.

[0059] The control unit 122 adjusts the processing of the treatment object on the stoker 30 so as to reduce the difference in the accumulation state of the treatment object among the multiple sections SC based on the above-mentioned relative relationship (the relative relationship of the accumulation state of the treatment object in the multiple sections SC) acquired by the accumulation state evaluation unit 120. The control unit 122 adjusts the processing of the treatment object on the stoker 30 so as to reduce the amount of treatment object on the section SC that is evaluated as having a relatively large amount of accumulated treatment object compared to the other sections among the multiple sections SC, for example.

[0060] In one example, the control unit 122 adjusts the grate speed of the movable grate 32 for each section SC by the drive device 33 so as to reduce the difference in the accumulation state of the materials to be treated among the multiple sections SC. Instead of or in addition to adjusting the grate speed, the control unit 122 adjusts the amount of air blown from the blower 40 by the valve 42 for each section SC so as to reduce the difference in the accumulation state of the materials to be treated among the multiple sections SC.

[0061] Fig. 10 is a block diagram illustrating a hardware configuration of the control device 100. The control device 100 is a control computer such as a programmable logic controller, and has a circuit 190 as shown in Fig. 10. The circuit 190 has a processor 191, a memory 192, a storage 193, an imaging control circuit 194, a grate control circuit 195, and a blower control circuit 196.

[0062] The storage 193 stores processing programs for causing the control device 100 to execute at least an image acquisition step, an area evaluation step, a distance distribution evaluation step, and an accumulation status evaluation step. These steps will be described later. The storage 193 stores, for example, processing programs for causing the control device 100 to configure each of the above-mentioned functional blocks. Specific examples of the storage 193 include a read-only memory, a non-volatile memory, a hard disk, and the like. The storage 193 may be a disk or a portable medium such as a USB memory.

[0063] The memory 192 temporarily stores the program loaded from the storage 193. A specific example of the memory 192 is a random access memory. The processor 191 executes the program loaded in the memory 192 to configure each of the above-mentioned functional blocks in the control device 100. The processor 191 appropriately stores the calculation results in the process of executing the program in the memory 192, and performs further calculations using the calculation results stored in the memory 192.

[0064] The imaging control circuit 194 acquires an image VI within the field of view from the imaging device 90 based on a command from the processor 191. The grate control circuit 195 controls the drive device 33 based on a command from the processor 191. The air blowing control circuit 196 controls the valve 42 based on a command from the processor 191.

[0065] [How to process the material] Next, a method for processing an object to be processed, which is executed in the stoker type processing device 1, will be described. This processing method includes at least a transporting step, an imaging step, an area evaluation step, a distance distribution evaluation step, and a deposition status evaluation step. The transporting step is a step of transporting the object to be processed in a transport direction D1 intersecting the vertical direction by a stoker 30 supporting the object to be processed. The imaging step is a step of imaging a field of view FV including a plurality of sections SC on the stoker 30 by an imaging device 90. When this imaging step is executed, the image acquisition step is executed. The image acquisition step is a step of the control device 100 acquiring an image VI within the field of view. The distance distribution evaluation unit 112 of the control device 100 may execute the image acquisition step.

[0066] The area evaluation step is a step of identifying a target area Z1 in which a processing target exists in the field of view image VI based on the field of view image VI obtained by imaging by the imaging device 90. The area evaluation step may be performed by the area evaluation unit 118 of the control device 100. The distance distribution evaluation step is a step of acquiring distance distribution information DI indicating the distance from the imaging device 90 to an object in the field of view FV. The distance distribution evaluation step may be performed by the distance distribution evaluation unit 112 of the control device 100.

[0067] The accumulation status evaluation step is a step of acquiring a relative relationship of accumulation statuses of the processing objects in the multiple sections SC based on accumulation status information DI1 obtained by combining the distance distribution information DI and the result of identifying the target area Z1. The accumulation status evaluation unit 120 of the control device 100 may execute the accumulation status evaluation step.

[0068] The imaging step, the area evaluation step, the distance distribution evaluation step, and the accumulation state evaluation step may be executed after the model construction step of constructing the distance estimation model M1, the area identification model M2, and the determination model M3 is executed. The model construction step may be executed at a stage before the stalker type treatment device 1 is operated, or at an initial stage immediately after the stalker type treatment device 1 is operated.

[0069] 11 is a flowchart showing a series of control processes executed by the control device 100 after the stoker type treatment device 1 is put into operation and after the model construction process is executed. While this series of control processes is being executed, the combustion of the treatment target on the stoker 30 and other processes are continued.

[0070] First, the control device 100 executes step S11. In step S11, for example, the control device 100 waits until it is time to evaluate the accumulation state of the processing target on the stoker 30. This evaluation timing is set in advance, and for example, is set so that the evaluation is repeated at a predetermined cycle.

[0071] Next, the control device 100 executes steps S12 and S13. In step S12, for example, the control device 100 causes the imaging device 90 to capture an image and acquires an in-field image VI from the imaging device 90. The control device 100 may have, as a functional block, an image acquisition unit that causes the imaging device 90 to capture an image and acquires an in-field image VI from the imaging device 90. In step S13, for example, the distance distribution evaluation unit 112 acquires distance distribution information DI indicating a distance to an object in the in-field image VI from the in-field image VI acquired in step S12. In one example, the distance distribution evaluation unit 112 inputs the in-field image VI to a distance estimation model M1 and acquires information output from the distance estimation model M1 at that time as distance distribution information DI.

[0072] Next, the control device 100 executes step S14. In step S14, for example, the area evaluation unit 118 identifies the target area Z1 in the visual field image VI where the processing target exists based on the visual field image VI obtained in step S12. In one example, the area evaluation unit 118 identifies the target area Z1 in the visual field image VI obtained in a state where the processing target exists on the stalker 30 without using other images (for example, without using the visual field image VI in an empty state where the processing target does not exist on the stalker 30 as a comparison target). The area evaluation unit 118 inputs the visual field image VI to the area identification model M2 and identifies the target area Z1 from the information output from the area identification model M2.

[0073] Next, the control device 100 executes step S15. In step S15, for example, the accumulation situation evaluation unit 120 acquires accumulation situation information DI1 by combining the distance distribution information DI acquired in step S13 with the result of identifying the target area Z1 in step S14. In one example, the accumulation situation evaluation unit 120 acquires the accumulation situation information DI1 by extracting information on pixels falling within the target area Z1 from the distance distribution information DI.

[0074] Next, the control device 100 executes step S16. In step S16, for example, the accumulation status evaluation unit 120 acquires a relative relationship of the accumulation status of the processing target in the multiple sections SC on the stoker 30 based on the accumulation status information DI1 acquired in step S15. In one example, the accumulation status evaluation unit 120 inputs the accumulation status information DI1 to a determination model M3, and acquires information output from the determination model M3 as information indicating the above-mentioned relative relationship. The accumulation status evaluation unit 120 may acquire information indicating one or more sections among the multiple sections SC in which the accumulation amount of the processing target is larger than the other sections, as information indicating the above-mentioned relative relationship.

[0075] Next, the control device 100 executes step S17. In step S17, for example, the control unit 122 controls the processing of the treatment object on the stoker 30 so as to reduce the difference in the accumulation state of the treatment object among the multiple sections SC based on the relative relationship obtained in step S16. In one example, the control unit 122 adjusts the grate speed by the drive unit 33 so as to reduce the difference in the accumulation amount of the treatment object among the multiple sections SC based on the relative relationship. The control unit 122 may control the drive unit 33 so as to increase the grate speed corresponding to the section SC that is evaluated to have a larger amount of treatment object than the other sections in the relative relationship obtained in step S16.

[0076] The control device 100 repeatedly executes the series of processes from steps S12 to S17 every time an evaluation timing occurs, thereby reducing bias in the accumulation state of the objects to be processed among the multiple sections SC.

[0077] [Variations] The series of processes shown in FIG. 11 is an example and can be modified as appropriate. In the series of processes described above, the control device 100 may execute one step and the next step in parallel, or may execute each step in an order different from the example described above. The control device 100 may omit any step. In the series of processes described above, the control device 100 may execute a step with a content different from the example described above.

[0078] 2 and 3, each of the multiple sections SC is divided into two groups arranged side by side. The control device 100 may acquire the relative relationship of the objects to be processed in the multiple sections SC for each of the left and right groups. In this case, the determination model M3 may be constructed by machine learning so as to output information indicating the above-mentioned relative relationship for each of the left and right groups.

[0079] The method of identifying the target region Z1 in the visual field image VI is not limited to the method of using the region identification model M2 by semantic segmentation. FIG. 12 shows a schematic diagram of a process of identifying the target region Z1 by a calculation different from the calculation using the region identification model M2. The region evaluation unit 118 repeats a process of comparing the standard deviation of pixel values ​​in a divided region M of the visual field image VI with a threshold value (hereinafter referred to as "threshold processing"). The divided region M is a part of the visual field image VI. The divided region M is set to a size that causes variation in pixel values ​​when the region contains a processing target object.

[0080] The area evaluation unit 118 performs the above threshold processing while sliding (scanning) the coordinates of the representative point of the divided area M to identify the target area Z1. The representative point of the divided area M is, for example, the center of the divided area M or the upper left corner of the divided area M. When sliding the coordinates of the representative point, the initial position of the divided area M is the upper left corner of the image in the field of view VI, and the coordinates are changed one by one in the horizontal direction on the image. After moving to the rightmost position in the horizontal direction, the vertical coordinate is changed by one, and then the coordinate is changed in the horizontal direction. The area evaluation unit 118 repeats sliding the coordinates of the representative point and the threshold processing until the divided area M moves to the lower right corner of the image in the field of view VI.

[0081] In FIG. 12, an example of a divided region M when a processing object is not included is indicated by "M1", and the divided region M when a processing object is not included has a small variation in pixel values. An example of a divided region M when a processing object is included is indicated by "M2", and the divided region M when a processing object is included has a large variation in pixel values. When the standard deviation of pixel values ​​in the divided region M is larger than a threshold value, the region evaluation unit 118 may determine that the processing object is present at the coordinates of the representative point of the divided region M. When the standard deviation of pixel values ​​in the divided region M is smaller than a threshold value, the region evaluation unit 118 may determine that the processing object is not present at the coordinates of the representative point of the divided region M.

[0082] The area evaluation unit 118 may identify the target area Z1 in the field of view image VI by a set of coordinates of the representative points whose standard deviation of pixel values ​​is determined to be greater than a threshold value. The threshold value used for comparison with the standard deviation may be set by confirming in advance the difference in the standard deviation of pixel values ​​depending on the presence or absence of a processing target. Even in the method using the threshold processing, the area evaluation unit 118 identifies the target area Z1 in the field of view image VI from the field of view image VI obtained in a state in which a processing target exists on the stalker 30, without using other images.

[0083] Instead of using the region identification model M2, the region evaluation unit 118 may identify the target region Z1 by executing a process of detecting edges in the in-field image VI. The region evaluation unit 118 may execute edge detection using the Canny method as the process of detecting edges. The region evaluation unit 118 executes edge detection using the Canny method by, for example, smoothing the image using a Gaussian filter or the like, cleaning out possible edge locations using a Sobel filter, and detecting edges using hysteresis in this order.

[0084] By executing the process of detecting edges, a portion that is a boundary between the target region Z1 and the non-target region Z2 in the visual field image VI can be detected as an edge. The region evaluation unit 118 may determine that the region surrounded by a set of coordinates detected as edges in the visual field image VI is the target region Z1. Even in the method using edge detection, the region evaluation unit 118 identifies the target region Z1 in the visual field image VI from the visual field image VI obtained in a state where the processing target object is present on the stalker 30, without using other images.

[0085] Instead of using the region identification model M2, the region evaluation unit 118 may identify the target region Z1 by evaluating the change in pixel value over time for each pixel in a plurality of images VI within the field of view. In one example, the region evaluation unit 118 acquires a plurality of images VI within the field of view obtained by continuing the imaging of the imaging device 90 for a predetermined time. The predetermined time may be several tens of seconds to a few minutes, for example, 30 to 90 seconds. The predetermined time may be shorter than the time from the above-mentioned evaluation timing to the next evaluation timing.

[0086] The area evaluation unit 118 may acquire multiple images VI within the visual field by having the imaging device 90 capture images every second during the predetermined time. Fig. 13 shows (N+1) images VI within the visual field obtained by repeatedly capturing images by the imaging device 90 during a period T corresponding to the predetermined time. The area evaluation unit 118 performs a process of comparing, for each pixel in the multiple images VI within the visual field, the standard deviation of the change in pixel value over time at that pixel with a threshold value.

[0087] When focusing on a certain pixel, if the pixel does not have a processing object, the time change of the pixel value is small, and as a result, the standard deviation of the pixel value is also small. On the other hand, if the pixel has a processing object, the time change of the pixel value is large, and as a result, the standard deviation of the pixel value is large. The area evaluation unit 118 may determine the last image VI(N) in the field of view obtained during the predetermined time as the image to be used to identify the target area Z1. The area evaluation unit 118 may identify the target area Z1 in the image VI(N) in the field of view by a set of coordinates of pixels determined to have a standard deviation in the time change of pixel value greater than a threshold value.

[0088] Even in a method that uses a process of comparing the standard deviation of the time change in pixel values ​​with a threshold value, the area evaluation unit 118 identifies the target area Z1 in the field of view image VI from (multiple) field of view images VI obtained when the processing target object is present on the stalker 30, without using any other images.

[0089] The method of evaluating the relative relationship from the deposition status information DI1 is not limited to the method of using the determination model M3, which is a trained model constructed by machine learning. A method of evaluating (obtaining) the relative relationship by a method other than the method of using the determination model M3 will be described with reference to Figs. 14(a), 14(b), and 15.

[0090] The deposition status evaluation unit 120 executes the following two processes. The first process is a process for calculating, for each pixel in the deposition status information DI1, three-dimensional coordinates consisting of the two-dimensional coordinates and distance (depth from the imaging device 90) of the pixel. The second process is a process for acquiring a relative relationship based on information indicating areas corresponding to multiple sections SC on a three-dimensional graph using the two-dimensional coordinates and the distance as variables, and the calculation results of the three-dimensional coordinates for each pixel in the deposition status information.

[0091] Here, the three-dimensional coordinates used in the above two processes are expressed as "(d2, z, d3)". d2 is the horizontal coordinate of the accumulation status information DI1 (two-dimensional image), and z is the vertical coordinate of the accumulation status information DI1. d3 is the distance from the imaging device 90 at the coordinate (d2, z), and can also be said to be the coordinate in the depth direction D3. In the various examples described above, the coordinates (d2, z) of each pixel may match between the image within the field of view VI, the distance distribution information DI, and the accumulation status information DI1. The unit of the two-dimensional coordinates may also be the unit of the distance from a predetermined reference position.

[0092] 14(a) shows a schematic representation of the three-dimensional coordinates of several pixels. The two-dimensional coordinates of the pixel indicated by "A" (hereinafter referred to as "pixel A") are (350, 2000), and the distance of pixel A in the distance distribution information DI is 2500. The two-dimensional coordinates of the pixel indicated by "B" (hereinafter referred to as "pixel B") are (400, 500), and the distance of pixel B in the distance distribution information DI is 500. The two-dimensional coordinates of the pixel indicated by "C" (hereinafter referred to as "pixel C") are (1200, 1500), and the distance of pixel C in the distance distribution information DI is 1500.

[0093] On the right side of Fig. 14(a), a three-dimensional graph (hereinafter, referred to as "three-dimensional graph Gr") is illustrated, in which the variables are two-dimensional coordinates on the image and the distance from the imaging device 90. This three-dimensional graph Gr is a graph having a three-dimensional Cartesian coordinate system, and the three axes represent distance in the width direction D2, the vertical direction (direction Z), and the depth direction D3. In the three-dimensional graph Gr shown in Fig. 14(a), the three-dimensional coordinates of each of pixel A, pixel B, and pixel C are plotted.

[0094] The left side of Fig. 14(b) illustrates deposition status information DI1, and the right side of Fig. 14(b) illustrates a three-dimensional graph Gr obtained by plotting the three-dimensional coordinates of each pixel in the distance distribution information DI on a graph. In Fig. 14(b), the darkened parts in the three-dimensional graph Gr are a set of points obtained by plotting the three-dimensional coordinates of all pixels (all pixels in an area having a pixel value) in the distance distribution information DI. This three-dimensional graph Gr makes it possible to show how much of the processing object is deposited at what position (i.e., the deposition status of the processing object).

[0095] Areas corresponding to a plurality of sections SC can be predefined on the three-dimensional graph Gr. FIG. 15 shows a schematic diagram of the result of defining the areas corresponding to a plurality of sections SC on the three-dimensional graph Gr. The control device 100 has, for example, an area information storage unit 132 as a functional block (see FIG. 4). The area information storage unit 132 stores area information indicating the areas corresponding to a plurality of sections SC on the three-dimensional graph Gr. The area information is predefined, for example, by an operator of the stalker-type processing device 1. In the area information, a three-dimensional range on the three-dimensional graph Gr is defined for each section SC.

[0096] In one example, the accumulation situation evaluation unit 120 calculates the three-dimensional coordinates (d2, z, d3) of each pixel from the two-dimensional coordinates and pixel values ​​of all pixels in the accumulation situation information DI1. Then, the accumulation situation evaluation unit 120 counts how many three-dimensional coordinates of pixels in the accumulation situation information DI1 are included for each section SC whose range on the three-dimensional graph is defined by the area information. After that, the accumulation situation evaluation unit 120 obtains the relative relationship of the accumulation situations of the processing object in the multiple sections SC from the relative difference in the number of pixels in the multiple sections SC.

[0097] For example, when the data plotted in the three-dimensional graph Gr shown in Fig. 14(b) is obtained and the number of pixels in each of the sections 1 to 5 shown in Fig. 15 is counted, the number of pixels in section 4 and section 5 is greater than the other sections. In this case, the accumulation status evaluation unit 120 evaluates that the processing target has accumulated in greater amounts in section 4 and section 5 than in the other sections.

[0098] As described above, in an image acquired by the imaging device 90 (camera), the size of an object on the image changes depending on the distance (depth) to the object. In order to avoid overestimating a processing target object at a certain location on the image and underestimating a processing target object at a certain distance, the accumulation status evaluation unit 120 may correct the number of pixels included in the section SC according to the distance in the depth direction D3. For example, the accumulation status evaluation unit 120 may add or multiply, for each section SC, the number of pixels included in the section by a correction constant according to the distance of the section in the depth direction D3. The correction constant is predetermined so that the value increases as the distance in the depth direction D3 increases.

[0099] The objects for evaluating the relative relationship of the accumulation state of the processing object are not limited to the multiple sections SC arranged in the conveying direction D1. Instead of or in addition to evaluating the relative relationship in the conveying direction D1, the accumulation state evaluation unit 120 may evaluate the relative relationship between two or more sections arranged in the width direction D2. In one example among the various examples described above, at least a part of the matters described in the other examples may be combined.

[0100] [Summary of this disclosure] The stoker type treatment apparatus (1) described above includes a stoker (30) that supports the object to be treated and transports the object to be treated in a transport direction (D1) that intersects the vertical direction, an imaging device (90) that images a field of view (FV) including multiple compartments (SC) on the stoker (30), an area evaluation unit (118) that identifies a target area (Z1) in the field of view image (VI) in which the object to be treated exists based on a two-dimensional field of view image (VI) obtained by imaging with the imaging device (90), a distance distribution evaluation unit (112) that acquires distance distribution information indicating the distance from the imaging device (90) to an object in the field of view (FV), and a deposition situation evaluation unit (120) that acquires the relative relationship of deposition situations of the object to be treated in the multiple compartments (SC) based on deposition situation information (DI1) obtained by combining the distance distribution information (DI) and the result of identifying the target area (Z1). The accumulation status information (DI1) obtained by combining the distance distribution information (DI) with the result of identifying the target area (Z1) includes information indicating at what position in the field of view (FV) the object to be treated is accumulated and to what extent compared to other positions. By acquiring the relative relationship of the accumulation status of the object to be treated in the multiple sections (SC) based on such accumulation status information (DI1), the accumulation status of the object to be treated on the stoker (30) can be easily obtained. Therefore, the stoker type treatment device (1) is useful for simplifying the process of acquiring the accumulation status of the object to be treated on the stoker.

[0101] In the stoker type treatment apparatus (1) described above, the multiple sections (SC) may be aligned in the conveying direction (D1). In this case, the difference in the accumulation state (the difference in the accumulation amount) of the objects to be processed in the transport direction (D1) can be reduced.

[0102] In the stalker-type treatment device (1) described above, the imaging device (90) may form an image of mid-infrared light incident from the field of view (FV) to generate a field of view image (VI). In this case, the imaging device (90) can acquire information that is not easily influenced by flames, water vapor, carbon dioxide gas, and the like generated by the combustion of the material to be treated. As a result, the accumulation state of the material to be treated on the stoker can be acquired with high accuracy.

[0103] In the stalker-type processing device (1) described above, the distance distribution evaluation unit (112) may acquire distance distribution information (DI) by inputting the in-field image (VI) to a distance estimation model constructed by machine learning so as to output information representing the distribution of distances to objects in a two-dimensional image in response to an input of the two-dimensional image. In this case, by using a trained model, it is not necessary to obtain distance information separately from the visual image (VI), which is useful for simplifying the device.

[0104] In the stoker-type treatment device (1) described above, the deposition situation evaluation unit (120) may acquire the above-mentioned relative relationship by inputting the deposition situation information (DI1) to a judgment model constructed by machine learning so as to output information indicating the relative relationship of the deposition situations of the treatment object in multiple sections (SC) in response to input information corresponding to the deposition situation information (DI1). In this case, since the accumulation state information (DI1) serving as input data is input to the model to obtain the above relative relationship, the calculation for obtaining the accumulation state is simplified.

[0105] In the stalker type treatment device (1) described above, the distance distribution information (DI) may include information indicating the distance for each pixel in the field of view image (VI). The accumulation status evaluation unit (120) may calculate, for each pixel in the accumulation status information (DI1), a three-dimensional coordinate consisting of the two-dimensional coordinate of the pixel and the distance, and obtain the relative relationship based on area information indicating regions corresponding to each of a plurality of sections (SC) on a three-dimensional graph (Gr) having the two-dimensional coordinate and the distance as variables, and on the calculation result of the three-dimensional coordinate for each pixel in the accumulation status information (DI1). In this case, since there is no need to create a determination model in order to obtain the above-mentioned relative relationship from the accumulation status information (DI1), the preparation work is simplified.

[0106] In the stalker-type processing device (1) described above, the area evaluation unit (118) may identify the target area (Z1) by inputting the image within the field of view (VI) into an area identification model constructed by semantic segmentation so as to output information indicating the result of classifying each pixel in the two-dimensional image into an area where the processing object exists and an area where the processing object does not exist in response to the input of a two-dimensional image. In this case, in order to identify the target area (Z1), no image is required other than the field of view image (VI) obtained when the stoker-type treatment device (1) is operating and there is a treatment target on the stoker (30). For example, it is possible to obtain an initial image corresponding to the field of view image (VI) before the stoker-type treatment device (1) is operated (when there is no treatment target on the stoker 30), and to calculate the difference between the field of view image (VI) and the initial image to identify the area where the treatment target exists, but there is no need to use such an initial image. Therefore, it is further useful for simplifying the process of acquiring the accumulation status of the treatment target on the stoker.

[0107] In the stalker-type processing device (1) described above, the area evaluation unit (118) may identify the target area (Z1) by performing a process of comparing the standard deviation of pixel values ​​in the divided area (M) of the field of view image (VI) with a threshold value while sliding the coordinates of the representative point of the divided area (M). Similarly, in this case, in order to specify the target area (Z1), no image is required other than the image within the field of view (VI) obtained when the stoker-type treatment device (1) is operating and the treatment target is on the stoker (30). Therefore, it is further useful for simplifying the process of acquiring the accumulation status of the treatment target on the stoker.

[0108] In the stalker-type processing device (1) described above, the area evaluation unit (118) may specify the target area (Z1) by executing a process of detecting edges in the in-field image (VI). Similarly, in this case, in order to specify the target area (Z1), no image is required other than the image within the field of view (VI) obtained when the stoker-type treatment device (1) is operating and the treatment target is on the stoker (30). Therefore, it is further useful for simplifying the process of acquiring the accumulation status of the treatment target on the stoker.

[0109] In the stalker-type processing device (1) described above, the area evaluation unit (118) may acquire a plurality of in-field images (VI(0) to VI(N)) including an in-field image (VI), the plurality of in-field images (VI(0) to VI(N)) being obtained by continuing imaging with the imaging device (90) for a predetermined period of time, and identify the target area (Z1) by performing a process of comparing, for each pixel in the plurality of in-field images (VI(0) to VI(N)), the standard deviation of the change in pixel value over time with a threshold value. In this case, in order to specify the target area (Z1), no images are required other than the multiple field-of-view images (VI) obtained when the stoker-type treatment device (1) is operating and there is a treatment target on the stoker (30). Therefore, this is even more useful in simplifying the process of acquiring the accumulation status of the treatment target on the stoker.

[0110] The stoker type treatment device (1) described above may further include a control unit (122) that adjusts the treatment of the treatment object on the stoker (30) so as to reduce the difference in the accumulation state of the treatment object among the multiple sections (SC) based on the relative relationship acquired by the accumulation state evaluation unit (120). In this case, the difference in the accumulation state of the material to be treated among the multiple sections (SC) is reduced, which is useful for improving the efficiency of treatment by the stoker-type treatment device (1).

[0111] The processing method for the object to be processed described above includes a transporting step of transporting the object to be processed in a transport direction (D1) that intersects the vertical direction by a stoker (30) that supports the object to be processed; an imaging step of imaging a field of view (FV) including multiple sections (SC) on the stoker (30) by an imaging device (90); an area evaluation step of identifying a target area (Z1) in the field of view image (VI) in which the object to be processed exists based on a two-dimensional field of view image (VI) obtained by imaging by the imaging device (90); a distance distribution evaluation step of acquiring distance distribution information indicating the distance from the imaging device (90) to an object in the field of view (FV); and a deposition situation evaluation step of acquiring the relative relationship of the deposition situations of the object to be processed in the multiple sections (SC) based on deposition situation information (DI1) obtained by combining the distance distribution information (DI) and the result of identifying the target area (Z1). This processing method, like the stoker-type processing apparatus (1) described above, is useful for simplifying the process of acquiring information on the accumulation status of materials to be processed on the stoker.

[0112] The processing program described above is a processing program for causing a computer to execute the following steps: an image acquisition process for acquiring a two-dimensional field of view image (VI) obtained by imaging a field of view (FV) including multiple sections (SC) on a stoker (30) that supports the object to be processed and transports the object to be processed in a transport direction (D1) that intersects the vertical direction with an imaging device (90); an area evaluation process for identifying a target area (Z1) in the field of view image (VI) in which the object to be processed exists based on the field of view image (VI); a distance distribution evaluation process for acquiring distance distribution information (DI) indicating the distance from the imaging device (90) to an object in the field of view (FV); and a deposition situation evaluation process for acquiring the relative relationship of the deposition situations of the object to be processed in the multiple sections (SC) based on deposition situation information (DI1) obtained by combining the distance distribution information (DI) and the result of identifying the target area (Z1). This processing program is useful for simplifying the process of acquiring information on the accumulation status of materials to be processed on the stoker, similar to the stoker-type processing apparatus (1) described above. [Explanation of symbols]

[0113] 1...stoker type processing device, 30...stoker, D1...transport direction, SC...section, 90...imaging device, FV...field of view, VI...image within field of view, 112...distance distribution evaluation unit, DI...distance distribution information, M1...distance estimation model, 118...area evaluation unit, Z1...target area, M2...area identification model, 120...accumulation situation evaluation unit, M3...judgment model.

Claims

1. A stoker that supports an object to be treated and transports the object to be treated in a transport direction that intersects with a vertical direction; An imaging device that images a field of view including a plurality of sections on the stoker; an area evaluation unit that identifies a target area in the field of view image where the processing target object exists based on a two-dimensional field of view image obtained by imaging with the imaging device; a distance distribution evaluation unit that acquires distance distribution information indicating distances from the imaging device to objects within the field of view; an accumulation status evaluation unit that acquires a relative relationship between accumulation statuses of the processing object in the plurality of sections based on accumulation status information obtained by combining the distance distribution information and the result of identifying the target area; A stoker-type treatment device comprising:

2. The stoker-type treatment device according to claim 1 , wherein the plurality of compartments are aligned in the conveying direction.

3. The stalker-type processing device according to claim 1 , wherein the imaging device forms an image of mid-infrared light incident from the field of view to generate the image within the field of view.

4. 2. The stalker-type processing device according to claim 1, wherein the distance distribution evaluation unit acquires the distance distribution information by inputting the image within the field of view to a distance estimation model constructed by machine learning so as to output information representing a distribution of distances to objects in a two-dimensional image in response to an input of the two-dimensional image.

5. The deposition situation evaluation unit acquires the relative relationship by inputting deposition situation information into a judgment model constructed by machine learning so as to output information indicating the relative relationship of the deposition situations of the treatment object in the multiple sections in response to input information corresponding to the deposition situation information.A stoker-type treatment device as described in any one of claims 1 to 4.

6. the distance distribution information includes information indicating the distance for each pixel in the image within the field of view, The deposition status evaluation unit is For each pixel in the accumulation status information, a three-dimensional coordinate is calculated, the three-dimensional coordinate being the two-dimensional coordinate of the pixel and the distance; The stoker-type treatment device according to any one of claims 1 to 4, wherein the relative relationship is acquired based on area information indicating areas corresponding to each of the plurality of sections on a three-dimensional graph having the two-dimensional coordinates and the distance as variables, and a calculation result of the three-dimensional coordinates for each pixel in the accumulation status information.

7. The area evaluation unit inputs the image within the field of view into an area identification model constructed by semantic segmentation so as to output information indicating the result of classifying each pixel in the two-dimensional image into an area where the processing object exists and an area where the processing object does not exist in response to an input of a two-dimensional image, thereby identifying the target area. The stalker-type processing device according to any one of claims 1 to 4.

8. The stalker-type processing device according to any one of claims 1 to 4, wherein the area evaluation unit identifies the target area by performing a process of comparing a standard deviation of pixel values ​​in a divided area of ​​the image within the field of view with a threshold value while sliding coordinates of a representative point of the divided area.

9. The stalker type processing device according to any one of claims 1 to 4, wherein the area evaluation unit identifies the target area by executing a process of detecting an edge in the image within the field of view.

10. The area evaluation unit is Acquire a plurality of in-field images including the in-field image, the plurality of in-field images being obtained by continuing imaging of the imaging device for a predetermined period of time; The stalker-type processing device according to any one of claims 1 to 4, wherein the target region is identified by performing a process of comparing a standard deviation in a time change of pixel value with a threshold value for each pixel in the multiple field-of-view images.

11. The stoker type treatment device according to any one of claims 1 to 4, further comprising a control unit that adjusts the treatment of the object on the stoker so as to reduce a difference in the deposition state of the object to be treated among the plurality of sections based on the relative relationship acquired by the deposition state evaluation unit.

12. A transport step of transporting the object to be treated in a transport direction intersecting a vertical direction by a stoker supporting the object to be treated; an imaging step of imaging a field of view including a plurality of sections on the stoker by an imaging device; a region evaluation step of identifying a target region in the field of view image where the processing target object exists based on a two-dimensional field of view image obtained by imaging with the imaging device; a distance distribution evaluation step of acquiring distance distribution information indicating distances from the imaging device to objects within the field of view; a deposition status evaluation step of acquiring a relative relationship of deposition statuses of the processing object in the plurality of sections based on deposition status information obtained by combining the distance distribution information and the result of identifying the target area; A method for treating an object to be treated, comprising:

13. an image acquiring step of acquiring a two-dimensional image of a visual field obtained by capturing an image of a visual field including a plurality of compartments on a stoker that supports an object to be processed and transports the object to be processed in a transport direction intersecting a vertical direction, using an imaging device; a region evaluation step of identifying a target region in the image of the field of view where the processing target object exists based on the image of the field of view; a distance distribution evaluation step of acquiring distance distribution information indicating distances from the imaging device to objects within the field of view; a deposition status evaluation step of acquiring a relative relationship of deposition statuses of the processing object in the plurality of sections based on deposition status information obtained by combining the distance distribution information and the result of identifying the target area; A processing program for causing a computer to execute the above.

Images