Information processing method, information processing device, and information processing program

Abstract

To provide an information processing method, an information processing apparatus, and an information processing program which can reduce processes relating to introduction of a discrimination algorithm for discriminating kinds of waste in a newly introduced facility and provide high accuracy in discrimination.SOLUTION: An information processing method of generating a discrimination algorithm for discriminating kinds of waste has a step of generating a first model serving as a discrimination algorithm for kinds of waste by carrying out machine learning of first image data obtained by capturing an inside of a waste pit of each of a plurality of facilities for reserving waste and first teacher data indicating waste in the image labelled with kind, and a step of generating, for the generated first model, a second model serving as a discrimination algorithm corresponding to waste of a second facility by additionally learning second image data obtained by capturing an inside of a waste pit in the second facility different from the plurality of facilities and second teacher data indicating waste in the image labelled with kinds.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing method, an information processing apparatus, and an information processing program for generating an identification algorithm for identifying the types of waste.

Background Art

[0002] In a waste treatment facility, various types of waste are carried into a garbage pit. After being stored in the garbage pit, these wastes are put into an incinerator and incinerated. The quality of the waste put into the incinerator affects combustion. In addition, the garbage pit also contains waste that, if put in directly, would cause equipment trouble. Therefore, in conventional facilities, an operator visually identifies the types of garbage in the garbage pit and performs operations such as stirring the waste with a crane and putting it into the incinerator so as to stabilize combustion and prevent equipment trouble.

[0003] The applicant of this application has already proposed a technique for automatically identifying waste using an identification algorithm (trained model) for identifying the types of waste stored in a garbage pit and performing crane operations (see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The inventors of the present application conducted intensive studies to find an improved technique for the garbage identification method using the above-described identification algorithm. As a result, the following findings were obtained. Note that the following findings are merely the triggers for the present invention and do not limit the present invention.

[0006] In other words, as shown in Figure 8, machine learning (supervised learning) requires a large amount of correct data for training. Therefore, conventional waste identification methods using identification algorithms (conventional models) require a large amount of image data and training data that must be newly prepared for each waste treatment facility.

[0007] However, when introducing identification algorithms in newly constructed facilities, the period from the start of waste acceptance to the official start of facility operation is limited, so it is necessary to collect image data, create training data, and train the algorithms within a short period of time.

[0008] Furthermore, even if image data is collected, training data is created, and training is performed within a short period, the identification accuracy of the identification algorithm will not improve for waste that appears infrequently in new facilities because there is insufficient training data.

[0009] Furthermore, since waste treatment facilities differ in the shape and dimensions of their waste pits, as well as in various environmental conditions (for example, the installation position, angle, and lighting conditions of cameras used for image acquisition), it is not possible to obtain sufficient identification accuracy by using an identification algorithm created using image data and training data from one facility at another facility.

[0010] The present invention has been made in consideration of the above points. The object of the present invention is to provide an information processing method, an information processing device, and an information processing program that can reduce the man-hours required to introduce an identification algorithm for identifying types of waste in newly introduced facilities, and that have high identification accuracy. [Means for solving the problem]

[0011] The information processing method according to the first aspect of the present invention is: The process involves generating a first model, which is a waste type identification algorithm, by machine learning using first image data, which consists of images taken inside waste pits at multiple facilities where waste is stored, and first training data, which consists of waste labeled by type within those images. The first generated model is further trained with second image data, which is an identification algorithm corresponding to the waste of the second facility, and second training data, which is a type of waste labeled in the image data. Includes.

[0012] In this configuration, before introducing the identification algorithm to a newly introduced facility (the second facility), it is possible to pre-train using a large amount of training data collected and generated at multiple facilities without time limitations. Furthermore, even for waste materials that appear infrequently at a single facility, sufficient training data can be used for training. In addition, since training can be performed using diverse training data that takes into account differences in the shape and dimensions of waste pits at each facility, as well as differences in various environmental conditions (for example, the installation position, angle, and lighting conditions of the image acquisition camera), an identification algorithm (first model) with robustness (the property of being able to perform stable and highly accurate identification for diverse data) can be obtained. Next, a second model is generated by further training the first model with training data collected and generated at the newly introduced facility (the second facility). According to the inventors' findings, the amount of training data used for additional training (training data collected and generated at the newly introduced facility) should be small compared to the amount of training data used to generate the first model (training data collected and generated at multiple facilities). A small amount makes the model less susceptible to environmental conditions at the newly introduced facility (such as the installation position, angle, and lighting conditions of the image acquisition camera) and future environmental changes (such as changes in the shape and color of garbage bags and age-related soiling of the pit sidewalls). However, if a type of waste not present in the first training data exists at the newly introduced facility, the training data for that waste can be included in the training data collected and generated at the newly introduced facility, thereby generating a second model capable of identifying it. This configuration allows for a model (second model) with high identification accuracy that corresponds to the newly introduced facility while maintaining a certain degree of robustness. Furthermore, it reduces the effort required to introduce an identification algorithm for identifying waste types at the newly introduced facility.

[0013] An information processing method according to a second aspect of the present invention is an information processing method according to a first aspect, The second image data and the second training data include in the image the side wall of the waste pit of the second facility and / or a crane that agitates or transports the waste. In this configuration, additional learning can be performed that takes into account the differences in the shape and dimensions of the waste pit and the crane at the newly introduced facility (the second facility). As a result, the accuracy of the second model in identifying the side walls and / or cranes is improved, and the accuracy of identifying the type of waste can also be improved compared to using data that only includes waste (i.e., data that does not include the side walls and / or cranes of the waste pit in the image).

[0014] An information processing method according to a third aspect of the present invention is an information processing method according to a first or second aspect, The amount of the second training data relative to the first training data is 30% or less.

[0015] An information processing method according to a fourth aspect of the present invention is an information processing method according to a third aspect, The amount of the second training data relative to the first training data is 15% or less.

[0016] The information processing method according to the fifth aspect of the present invention is the information processing method according to the fourth aspect, The amount of the second training data relative to the first training data is 5% or less.

[0017] The information processing method according to the sixth aspect of the present invention is an information processing method according to any one of the first to fifth aspects, The aforementioned discrimination algorithm includes one or more of the following: linear regression, Boltzmann machines, neural networks, support vector machines, Bayesian networks, sparse regression, decision trees, statistical estimation using random forests, reinforcement learning, and deep learning.

[0018] The information processing method according to the seventh aspect of the present invention is an information processing method according to any one of the first to sixth aspects, The second training data includes labeled images of the entire interior of the waste pit of the second facility. In this configuration, additional learning can be performed that takes into account information about how the images inside the waste pit appear (field of view and angle of view) obtained by the imaging device installed in the newly introduced facility (the second facility), thereby improving the accuracy of waste type identification by the second model.

[0019] An information processing method according to the eighth aspect of the present invention is an information processing method according to any one of the first to sixth aspects, The second training data includes a portion of an image taken inside the waste pit of the second facility, with only that portion labeled.

[0020] This approach significantly reduces the amount of training data and the effort required to create it compared to labeling the entire image.

[0021] The information processing method according to the ninth aspect of the present invention is the information processing method according to any one of the first to eighth aspects, and the second image data used in the step of generating the second model is obtained by inputting image data captured inside the trash pit of the second facility into the first model to identify the types of waste, and then selecting and using, from among those image data, image data with an identification accuracy lower than a predetermined criterion (first criterion).

[0022] According to such an aspect, in the second facility, it is possible to confirm the types of waste for which sufficient identification accuracy cannot be obtained with the first model, and perform additional learning mainly using the image data and teacher data including such types. As a result, efficient collection and learning of learning data become possible.

[0023] The information processing method according to the tenth aspect of the present invention is the information processing method according to any one of the first to tenth aspects, and in the step of generating the first model, information on the shooting conditions and / or shooting environment at each facility is learned together.

[0024] According to such an aspect, even if the shooting conditions (such as the resolution, exposure time, gain, focus, etc. of the camera) or shooting environment (such as the amount of natural light depending on the time and weather, the ON / OFF of lighting, etc.) change due to troubles or the like, it can still be operated.

[0025] The information processing method according to the eleventh aspect of the present invention is the information processing method according to any one of the first to tenth aspects, and the first image data is obtained by performing at least one correction of hue, brightness, or saturation on the image data captured inside the trash pit of each facility, with reference to an image of a color chart for image correction that is common among the plurality of facilities.

[0026] This configuration allows for correction of differences in color tone, brightness, and saturation of images captured at multiple facilities with different lighting and natural light conditions, thereby improving the recognition accuracy of the recognition algorithm.

[0027] An information processing method according to the twelfth aspect of the present invention is an information processing method according to any of the first to ten aspects, The first image data is obtained by capturing the inside of the waste pit at each facility, along with a common color chart for image correction used across the multiple facilities.

[0028] This method also allows for correction of differences in color tone, brightness, and saturation of images captured at multiple facilities with different lighting and natural light conditions, thereby improving the recognition accuracy of the recognition algorithm.

[0029] An information processing method according to the 13th aspect of the present invention is an information processing method according to any of the 1st to 12th aspects, The second image data includes a composite image of the waste in the waste pits of the multiple facilities, or the waste in the waste pits of other facilities different from both the multiple facilities and the second facility, onto a rendering image of the side wall of the waste pit and / or a crane used for agitating or transporting the waste, which was created based on the three-dimensional design data of the second facility.

[0030] In this configuration, image data regarding the appearance of the pit, crane, etc., can be created before the completion of pit construction and crane installation at the newly introduced facility (the second facility), and can be used for training.

[0031] An information processing method according to the 14th aspect of the present invention is an information processing method according to any of the 1st to 13th aspects, After the second model is put into operation, the identification accuracy during operation is periodically monitored, and if the identification accuracy falls below a predetermined standard (second standard), the second model is updated by additionally training it with the image data of the waste at that time and training data in which the waste in the image is labeled by type.

[0032] This configuration makes it possible to respond to changes in the types of waste and changes in the proportion of each type of waste.

[0033] An information processing method according to the 15th aspect of the present invention is an information processing method according to any of the 1st to 14th aspects, The process further includes the step of generating a third model, which is an identification algorithm corresponding to the waste of the third facility, by further training the generated second model with third image data, which is image data taken inside the waste pit of a third facility different from both the plurality of facilities and the second facility, and third training data, which is data in which the waste in the image is labeled by type.

[0034] In this configuration, the model (second model) obtained by additionally training the first training data with the second training data can be used as a base model for further training using the third training data from another facility (third facility), and thus the accuracy of identifying the type of waste can be expected to improve sequentially.

[0035] An information processing device according to the 16th aspect of the present invention is: A first model generation unit generates a first model, which is a waste type identification algorithm, by machine learning with first image data, which is image data taken inside the waste pits of multiple facilities where waste is stored, and first training data, which is data in which the waste in the image is labeled by type. A second model generation unit generates a second model, which is an identification algorithm corresponding to the waste of the second facility, by further training the generated first model with second image data taken inside a waste pit of a second facility different from the aforementioned multiple facilities, and second training data in which the waste in the image is labeled by type. It is equipped with.

[0036] The information processing program according to the 17th aspect of the present invention is: On the computer, The process involves generating a first model, which is a waste type identification algorithm, by machine learning using first image data, which consists of images taken inside waste pits at multiple facilities where waste is stored, and first training data, which consists of waste labeled by type within those images. The first generated model is further trained with second image data, which is an identification algorithm corresponding to the waste of the second facility, and second training data, which is a type of waste labeled in the image data. Make it run.

[0037] The information processing method according to the 18th aspect of the present invention is: The first model is a waste type identification algorithm generated by machine learning using first image data taken inside the waste pits of multiple facilities where waste is stored, and first training data in which the waste in the images is labeled by type. The second model is an identification algorithm corresponding to the waste of the second facility, generated by further training using second image data taken inside the waste pit of a second facility different from the multiple facilities, and second training data in which the waste in the images is labeled by type. The second model is then used to identify the type of waste stored in the waste pit of the second facility, taking new image data taken inside the waste pit of the second facility as input. Includes.

[0038] An information processing device according to the 19th aspect of the present invention is: A waste type identification unit uses a second model, which is an identification algorithm for waste at a second facility, to identify the type of waste stored in a waste pit, using new image data taken from the waste pit of the second facility as input. This second model is generated by machine learning a first model, which is a waste type identification algorithm, using a second model, which is an identification algorithm for waste at a second facility, to be further trained using a second model, which is a waste type identification algorithm for waste at a second facility, using a second model, which is a waste type identification algorithm for waste at a second facility, to identify the type of waste stored in the waste pit, using a first model, which is a waste type identification algorithm, generated by machine learning using a first model, which is a waste type identification algorithm, using a second model, which is a waste type identification algorithm for waste at a second facility, generated by further training using a second model, which is a waste type identification algorithm for waste at a second facility, using a second model, which is a waste type identification algorithm for waste at a second facility, to identify the type of waste stored in the waste pit, using new image data taken from the waste pit of the second facility as input. It is equipped with.

[0039] An information processing program according to a 20th aspect of the present invention is: On the computer, The first model is a waste type identification algorithm generated by machine learning using first image data taken inside the waste pits of multiple facilities where waste is stored, and first training data in which the waste in the images is labeled by type. The second model is an identification algorithm corresponding to the waste of the second facility, generated by further training using second image data taken inside the waste pit of a second facility different from the multiple facilities, and second training data in which the waste in the images is labeled by type. The second model is then used to identify the type of waste stored in the waste pit of the second facility, taking new image data taken inside the waste pit of the second facility as input. Make it run. [Effects of the Invention]

[0040] According to the present invention, the man-hours required to introduce an identification algorithm for identifying the type of waste in newly introduced facilities can be reduced. [Brief explanation of the drawing]

[0041] [Figure 1] Figure 1 is a schematic diagram showing the configuration of a waste treatment facility according to one embodiment. [Figure 2]Figure 2 is a block diagram showing the configuration of an information processing device according to one embodiment. [Figure 3] Figure 3 is a flowchart showing an example of an information processing method using an information processing device according to one embodiment. [Figure 4] Figure 4 shows an example of image data obtained by capturing video footage inside a waste pit. [Figure 5] Figure 5 shows an example of training data in which the type of waste and other identifiable objects are labeled on image data obtained by capturing video inside a waste pit. [Figure 6] Figure 6 shows an example of data in which the identification results from the waste type identification unit are superimposed on image data obtained by capturing video from inside the waste pit. [Figure 7] Figure 7 is a map showing the proportion of different types of waste in each area of ​​the waste pit. [Figure 8] Figure 8 is a conceptual diagram illustrating a conventional method for generating identification algorithms. [Figure 9] Figure 9 is a conceptual diagram showing a method for generating an identification algorithm according to one embodiment. [Figure 10] Figure 10 is a block diagram showing the configuration of an information processing device according to one modified example of one embodiment. [Modes for carrying out the invention]

[0042] Embodiments of the present invention will be described in detail below with reference to the attached drawings. In the following description and the drawings used therein, the same reference numerals will be used for parts that can be identically configured, and redundant explanations will be omitted.

[0043] Figure 1 is a schematic diagram showing the configuration of a waste treatment facility 100 (hereinafter sometimes referred to as the second facility) according to one embodiment.

[0044] As shown in Figure 1, the waste treatment facility 100 includes a platform 21 where transport vehicles (garbage trucks) 22 loaded with waste are parked, a waste pit 3 where waste introduced from the platform 21 is stored, a crane 5 for agitating and transporting the waste stored in the waste pit 3, a hopper 4 into which the waste transported by the crane 5 is introduced, an incinerator 1 for burning the waste introduced from the hopper 4, and a waste heat boiler 2 for recovering waste heat from the exhaust gas generated in the incinerator 1. The type of incinerator 1 is not limited to a stoker furnace as shown in Figure 1, but also includes a fluidized bed furnace. Furthermore, the structure of the waste pit 3 is not limited to a single-stage pit as shown in Figure 1, but also includes a two-stage pit in which the waste pit is divided into an input section and a storage section. The waste treatment facility 100 is also equipped with a crane control device 50 for controlling the operation of the crane 5 and a combustion control device 20 for controlling the combustion of waste in the incinerator 1.

[0045] The waste, loaded onto the transport vehicle 22, is fed from the platform 21 into the waste pit 3 and stored in the waste pit 3. The waste stored in the waste pit 3 is agitated by the crane 5 and then transported by the crane 5 to the hopper 4, which is then fed into the incinerator 1, where it is incinerated and processed.

[0046] As shown in Figure 1, the waste treatment facility 100 is equipped with an imaging device 6 (hereinafter also referred to as a camera) that captures images of the inside of the waste pit 3, and an information processing device 10 that identifies the type of waste in the waste pit 3.

[0047] The imaging device 6 is positioned above the waste pit 3 and, in the illustrated example, is fixed to the rails of the crane 5, enabling it to image the waste stored in the waste pit 3 from above. Figure 4 shows an example of image data obtained by imaging the inside of the waste pit 3 with the imaging device 6.

[0048] The imaging device 6 may be an RGB camera that outputs shape and color image data of waste as imaging results, a near-infrared camera that outputs near-infrared image data of waste as imaging results, a 3D camera or RGB-D camera that captures three-dimensional image data of waste as imaging results, or a combination of two or more of these.

[0049] Next, the configuration of the information processing device 10, which identifies the type of waste stored in the waste pit 3, will be described. Figure 2 is a block diagram showing the configuration of the information processing device 10. The information processing device 10 may be composed of one or more computers.

[0050] As shown in Figure 2, the information processing device 10 includes a control unit 11, a storage unit 12, and a communication unit 13. Each unit is connected to the others via a bus so that they can communicate with each other.

[0051] Of these, the communication unit 13 is a communication interface between the imaging device 6, the crane control device 50, and the combustion control device 20 and the information processing device 10. The communication unit 13 transmits and receives information between the imaging device 6, the crane control device 50, and the combustion control device 20 and the information processing device 10.

[0052] The storage unit 12 is a fixed data storage device, such as a hard disk. The storage unit 12 stores various types of data handled by the control unit 11. The storage unit 12 also stores the identification algorithm 12a generated by the first model generation unit 11a1 and the second model generation unit 11a2, which will be described later, and the image data 12b acquired by the image data acquisition unit 11b, which will be described later.

[0053] The control unit 11 is a control means that performs various processing of the information processing device 10. As shown in Figure 2, the control unit 11 includes a first model generation unit 11a1, a second model generation unit 11a2, an image data acquisition unit 11b, a type identification unit 11c, a plant control unit 11d, a fall detection unit 11e, and a foreign object input detection unit 11f. Each of these units may be realized by a processor in the information processing device 10 executing a predetermined program, or it may be implemented in hardware.

[0054] The first model generation unit 11a1 generates a waste type identification algorithm 12a (first model) by machine learning using first image data captured inside the waste pits of multiple facilities where waste is stored (waste treatment facilities different from the second facility 100) and first training data in which the waste in the images is labeled by type.

[0055] The first model generation unit 11a1 may generate an identification algorithm 12a (first model) that identifies the types of identification objects other than waste, in addition to the types of waste stored in the waste pits, by performing machine learning on first image data captured from the waste pits of multiple facilities and first training data in which the waste in the images is labeled by type, as well as other identification objects in the images.

[0056] The discrimination algorithm 12a may specifically include one or more of the following: linear regression, Boltzmann machines, neural networks, support vector machines, Bayesian networks, sparse regression, decision trees, statistical estimation using random forests, reinforcement learning, and deep learning.

[0057] The first training data may be created, for example, by skilled operators at each facility visually identifying waste and non-waste identifiable objects in the first image data captured inside the waste pit and labeling them by type. The types of waste and non-waste identifiable objects may be labeled, for example, by being superimposed on the first image data as type-specific layers.

[0058] The types of waste labeled in the first training data may include one or more of the following: unbroken garbage bags, paper waste, pruned branches, futons, sludge, bulky crushed waste, cardboard, burlap sacks, paper bags, and bottom waste (waste located near the bottom of garbage pit 3 that is compressed by the waste above and has a high moisture content). The types of waste labeled in the first training data may also include unintended waste (anomalous materials) that should not enter the garbage pit but may enter. Examples of anomalous materials include items that should not be incinerated, specifically fluorescent lamps, mercury-contaminated waste, and explosives such as cylinders, cans, and oil tanks. Furthermore, the types of waste labeled in the first training data may include one or more of the following: wood chips, textile waste, clothing waste, plastic waste, animal residue, animal carcasses, kitchen waste, vegetation, soil, medical waste, incinerator ash, bicycles, chests of drawers, beds, shelves, desks, chairs, agricultural vinyl, PET bottles, styrofoam, meat and bone meal, agricultural products, pottery, glass waste, metal scrap, rubble, concrete waste, tatami mats, bamboo, straw, and activated carbon.

[0059] The types of non-waste identifiable objects labeled in the first training data may include one or more of the following: beams of each facility, side walls of waste pits, cliffs of waste piles stored in waste pits (areas on the cliffs of waste piles that are so dark that the type of waste cannot be visually identified), and cranes used for agitating or transporting waste. Furthermore, the types of non-waste identifiable objects labeled in the first training data may include one or both of the following: workers and transport vehicles. Additionally, the types of non-waste identifiable objects labeled in the training data may include one or more of the following: walls, columns, floors, windows, ceilings, doors, stairs, girders (structures that suspend and move cranes), walkways, partition walls of waste pits, waste input hoppers, transport doors, workers, and transport vehicles.

[0060] The first model generation unit 11a1 may, when generating the first model by machine learning the first image data and the first training data, also learn information about the shooting conditions (camera resolution, exposure time, gain, focus, etc.) and / or the shooting environment (amount of natural light depending on the time of day and weather, ON / OFF status of lighting, etc.) at each facility. This allows the system to operate even if the shooting conditions or environment change due to troubles or other issues.

[0061] The first image data may be uncorrected image data captured inside the waste pit of each facility, to which at least one of the following corrections—tone, brightness, and saturation—has been applied, using an image of a common image correction color chart captured across multiple facilities as a reference. Alternatively, the image data may be captured inside the waste pit of each facility together with a common image correction color chart captured across multiple facilities. This allows the first model generation unit 11a1 to correct for differences in tone, brightness, and saturation of images captured at multiple facilities with different lighting and natural light conditions when generating the first model by machine learning with the first image data and the first training data, thereby improving the recognition accuracy of the recognition algorithm 12a.

[0062] The second model generation unit 11a2 generates an identification algorithm 12a (second model) corresponding to the waste of the second facility by further training the identification algorithm 12a (first model) generated by the first model generation unit 11a1 with second image data captured inside the waste pit of the second facility 100 and second training data in which the waste in the image is labeled by type.

[0063] The second model generation unit 11a2 may generate an identification algorithm 12a (second model) that identifies not only the types of waste stored in the waste pit of the second facility 100, but also the types of other objects to be identified besides waste, by further training the identification algorithm 12a (first model) generated by the first model generation unit 11a1 with second image data captured inside the waste pit of the second facility 100, and second training data in which the waste in the image is labeled by type, as well as other objects to be identified besides waste in the image.

[0064] According to the inventors' findings, the amount of training data used to create the second model (amount of second training data) can be small compared to the amount of training data required to create the first model (amount of first training data), preferably 30% or less, more preferably 15% or less, and even more preferably 5% or less. If the training data for the second facility 100 is small, it becomes less susceptible to environmental conditions of the newly introduced facility (installation position, angle, lighting conditions of the image acquisition camera, etc.) and future environmental changes (changes in the shape and color of garbage bags, age-related soiling of the pit side walls, etc.). However, if a type of waste that did not exist when the first model was generated is present in the newly introduced facility, the training data for that waste can be included in the training data collected and generated at the newly introduced facility, thereby generating a second model that can identify it. By configuring it in this way, a highly accurate identification algorithm 12a (second model) that corresponds to the newly introduced facility can be obtained while maintaining a certain degree of robustness, and the man-hours required for its generation can be reduced.

[0065] The second training data may be created, for example, by a skilled operator operating the second facility 100 visually identifying waste and non-waste identifiable objects from the second image data captured inside the waste pit 3 and labeling them by type. The types of waste and non-waste identifiable objects may be labeled, for example, by being superimposed on the second image data as type-specific layers.

[0066] The types of waste labeled in the second training data may include one or more of the following: unbroken garbage bags, paper waste, pruned branches, futons, sludge, bulky crushed waste, cardboard, burlap sacks, paper bags, and bottom waste (waste located near the bottom of garbage pit 3 that is compressed by the waste above and has a high moisture content). In addition, the types of waste labeled in the second training data may include unplanned waste (anomalous materials) that should not enter garbage pit 3 but may enter. Examples of anomalous materials include items that should not be incinerated, specifically, fluorescent lamps, mercury-contaminated waste, and explosives such as cylinders, cans, and oil tanks. Furthermore, the types of waste labeled in the second training data may include one or more of the following: wood chips, textile waste, clothing waste, plastic waste, animal residue, animal carcasses, kitchen waste, vegetation, soil, medical waste, incinerator ash, bicycles, chests of drawers, beds, shelves, desks, chairs, agricultural vinyl, PET bottles, styrofoam, meat and bone meal, agricultural products, pottery, glass waste, metal scrap, rubble, concrete waste, tatami mats, bamboo, straw, and activated carbon. Note that the types of waste labeled in the second training data do not need to be the same as the types of waste labeled in the first training data. As mentioned above, if a type of waste that does not exist in the first training data is present in the newly introduced facility, including that type of waste as a type of waste labeled in the second training data will allow for the generation of a second model that can identify it. Also, if there are types of waste present in the first training data that do not exist in the newly introduced facility, that type of waste does not need to be included as a type of waste labeled in the second training data.

[0067] The types of identifiable objects other than waste that are labeled in the second training data may include one or more of the following: beams of the second facility 100, side walls of the waste pit 3, cliffs of the waste piles stored in the waste pit 3 (the cliffs of the waste piles where the type of waste is too dark to be visually identified), and cranes 5 used for agitating or transporting the waste. In addition, the types of identifiable objects other than waste that are labeled in the second training data may include one or both of the following: workers and transport vehicles. In addition, the types of identifiable objects other than waste that are labeled in the training data may include one or more of the following: walls, columns, floors, windows, ceilings, doors, stairs, girders (structures that suspend and move cranes 5), walkways of the second facility 100, partition walls of the waste pit 3, waste input hoppers, transport doors, workers, and transport vehicles. Furthermore, the types of non-waste objects to be identified and labeled in the second training data do not need to be the same as the types of non-waste objects to be identified and labeled in the first training data. If there are types of non-waste objects to be identified and not present in the first training data that exist in the newly introduced facility, including those types of objects in the second training data will allow for the generation of a second model capable of identifying them. Also, if there are types of non-waste objects to be identified and not present in the first training data that do not exist in the newly introduced facility, those types of objects do not need to be included in the second training data.

[0068] Figure 5 shows an example of second training data in which the types of waste and non-waste identifiable objects are labeled on image data captured inside the waste pit 3 of the second facility 100. In the example shown in Figure 5, the image data captured inside the waste pit 3 is labeled by type as waste, including unbroken garbage bags, pruned branches, and futons, while the crane 5, the cliff of the waste pile, the side wall of the waste pit 3, and the floor of the second facility 100 are labeled by type as non-waste identifiable objects.

[0069] The second training data may include images of the waste pit 3 of the second facility 100 with the entire image labeled, or it may include images of the waste pit 3 of the second facility 100 with a portion cropped and only that cropped portion labeled. Including training data with the entire image labeled allows for additional learning that takes into account information about how the images of the waste pit look (field of view and angle of view) obtained by the imaging device installed in the newly introduced facility (the second facility), thereby improving the accuracy of waste type identification by the second model. Furthermore, including training data with partial labeling significantly reduces the amount of training data and the effort required to create it compared to labeling the entire image. In addition, the second image data and the second training data may include the side walls and / or crane 5 of the waste pit 3 of the second facility 100 in the image. Because the second training data includes the side walls of the waste pit of the second facility 100 and / or the crane used for agitating or transporting the waste in its images, additional learning can be performed that takes into account the differences in the shape and dimensions of the waste pit and the crane at the newly introduced facility (the second facility). As a result, the accuracy of the second model in identifying the side walls and / or the crane is improved, and the accuracy of identifying the waste can also be improved compared to using data containing only waste (i.e., data that does not include the side walls and / or the crane of the waste pit in the images).

[0070] The second image data may be obtained by inputting image data captured inside the waste pit 3 of the second facility 100 into the first model to identify the type of waste, and then selecting image data with a lower identification accuracy than a predetermined standard from among those image data, or it may be any image data that is not subject to a predetermined standard (i.e., the image data itself captured inside the waste pit 3 of the second facility 100). If the second image data is obtained by inputting image data captured inside the waste pit 3 of the second facility 100 into the first model to identify the type of waste, and then selecting image data with a low identification accuracy from among those image data, then at the second facility 100, the types of waste for which the first model cannot obtain sufficient identification accuracy can be identified, and additional learning can be performed mainly using image data and training data that include those types, thereby enabling efficient collection of training data and learning.

[0071] The second image data may include a composite image of a rendering image of the side wall and / or crane 5 of the waste pit 3, created based on the 3D design data of the second facility 100, and an image of waste in the waste pit of the multiple facilities, or waste in the waste pit of another facility different from both the multiple facilities and the second facility. In this case, image data regarding the appearance of the pit 3, crane 5, etc., can be created and used for training before the completion of pit construction and crane installation at the newly introduced facility (second facility 100).

[0072] The image data acquisition unit 11b acquires new image data from the imaging device 6, which is images taken inside the waste pit 3 of the second facility 100. The new image data acquired by the image data acquisition unit 11b is stored in the storage unit 12.

[0073] The type identification unit 11c takes the new image data acquired by the image data acquisition unit 11b as input and uses the identification algorithm 12a (second model) generated by the second model generation unit 11a2 to identify the type of waste stored in the waste pit 3 of the second facility 100.

[0074] The type identification unit 11c may use the identification algorithm 12a (second model) with the new image data acquired by the image data acquisition unit 11b as input to identify the types of waste stored in the waste pit 3 of the second facility 100, as well as the types of other objects to be identified. Figure 6 shows an example of data in which the identification results by the type identification unit 11c are superimposed on image data obtained by capturing video of the inside of the waste pit 3 of the second facility 100. In the example shown in Figure 6, the waste identified by the type identification unit 11c (unbroken bagged waste, pruned branches) and other objects to be identified (crane 5, cliff of the waste pile, side wall of the waste pit 3, floor of the second facility 100) are superimposed on the image data by type.

[0075] As shown in Figure 7, the type identification unit 11c may generate a map that displays the ratio of different types of waste stored in the waste pit 3 of the second facility 100 for each area, based on the identification results. In the example shown in Figure 7, the waste pit 3 is divided into a 5x4 grid, and the ratio of different types of waste identified by the type identification unit 11c is displayed for each area.

[0076] The plant control unit 11d controls the waste treatment facility (second facility) 100 based on the identification result of the type identification unit 11c.

[0077] In the example shown in Figure 1, the plant control unit 11d includes a crane control unit 11d1 that transmits the identification result of the type identification unit 11c (i.e., information on the type of waste identified from the image data) to a crane control device 50 that controls a crane 5 that agitates or transports waste, and a combustion control unit 11d2 that transmits the identification result of the type identification unit 11c (i.e., information on the type of waste identified from the image data) to a combustion control device 20 that controls the combustion of waste.

[0078] In the example shown in Figure 1, the plant control unit 11d includes both the crane control unit 11d1 and the combustion control unit 11d2, but it is not limited to this, and may include only one of the crane control unit 11d1 or the combustion control unit 11d2.

[0079] The crane control unit 11d1 transmits to the crane control device 50, for example, a map (see Figure 7) that displays the ratio of different types of waste stored in the waste pit 3 for each area, based on the identification results of the type identification unit 11c. Based on the map received from the crane control unit 11d1, the crane control device 50 operates the crane 5 to agitate the waste in the waste pit 3 so that the ratio of different types of waste becomes equal in all areas.

[0080] The combustion control unit 11d2 transmits to the combustion control device 20 a map (see Figure 7) that displays the ratio of different types of waste stored in the waste pit 3 for each region, based on the identification results of the type identification unit 11c. Based on the map received from the combustion control unit 11d2, the combustion control device 20 determines the ratio of different types of waste that are grabbed by the crane 5 and transported together from the waste pit 3 to the hopper 4, and controls the combustion of the waste according to the ratio of waste that is fed together into the incinerator 1 via the hopper 4 (for example, by controlling the stoker feed rate and the amount of air supplied).

[0081] The fall detection unit 11e detects when a worker or transport vehicle falls from the platform 21 into the waste pit 3 based on the identification result of the type identification unit 11c (i.e., information of the worker or transport vehicle identified from the image data). The fall detection unit 11e may also transmit the identification result of the type identification unit 11c (i.e., information of the worker or transport vehicle identified from the image data) to a fall detection device (not shown) that detects the presence of a worker or transport vehicle in the waste pit 3 where the waste is stored. Based on the identification result of the type identification unit 11c transmitted from the fall detection unit 11e, the fall detection device (not shown) may issue an alarm or operate the crane 5 or rescue equipment (e.g., a gondola) (not shown) to rescue the worker.

[0082] The foreign object input detection unit 11f detects abnormal objects that have been introduced into the waste pit 3 based on the identification result of the type identification unit 11c (i.e., information on the type of waste identified from the image data). Here, "abnormal object" refers to unplanned waste that is not intended to enter the waste pit 3 but may enter, such as items that should not be incinerated, specifically, fluorescent lamps, mercury-contaminated waste, and explosives such as cylinders, cans, and oil tanks. The foreign object input detection unit 11f may also transmit the identification result of the type identification unit (i.e., information on the type of waste identified from the image data) to a foreign object detection device (not shown) that detects abnormal objects introduced into the waste pit 3 where waste is stored. The foreign object detection device (not shown) refers to a database that stores the company or vehicle that introduced the waste into the waste pit 3 along with time information, and identifies the company or vehicle that introduced the foreign object into the waste pit 3 based on the identification result of the type identification unit 11c transmitted from the foreign object input detection unit 11f.

[0083] Next, an example of an information processing method using the information processing device 10 with the above configuration will be described. Figure 3 is a flowchart of an example of an information processing method.

[0084] As shown in Figure 3, first, the first model generation unit 11a1 generates a waste type identification algorithm 12a (first model) by machine learning using first image data captured inside the waste pits of multiple facilities where waste is stored (waste treatment facilities different from the second facility 100) and first training data in which the waste in the images is labeled by type (step S11).

[0085] Before being introduced to the second facility 100, the first model generation unit 11a can be pre-trained using a large amount of training data collected and generated at multiple facilities without time limitations. Furthermore, even waste items that appear infrequently at a single facility can be trained using a sufficient amount of training data. In addition, since training can be performed using diverse training data that takes into account differences in the shape and dimensions of waste pits at each facility, as well as differences in various environmental conditions (for example, the installation position, angle, and lighting conditions of the image acquisition camera), a robust identification algorithm 12a (first model) can be obtained.

[0086] Next, the second model generation unit 11a2 generates an identification algorithm 12a (second model) corresponding to the waste of the second facility by further training the identification algorithm 12a (first model) generated by the first model generation unit 11a1 with second image data captured inside the waste pit of the second facility 100 and second training data in which the waste in the image is labeled by type (step S12).

[0087] Here, the amount of training data used for additional training (second training data) should be small compared to the amount of training data used when generating the first model (first training data). A small amount will make the model less susceptible to environmental conditions of newly introduced facilities (such as the installation position, angle, and lighting conditions of the image acquisition camera) and future environmental changes (such as changes in the shape and color of garbage bags and age-related dirt on the pit side walls).

[0088] However, if a type of waste that does not exist in the first training data is present in the second facility 100, it is advisable to include training data for that waste in the second training data in order to generate an identification algorithm 12a (second model) that can identify it.

[0089] By configuring it in this way, a second model with high identification accuracy corresponding to the second facility 100 can be obtained while maintaining a certain degree of robustness. In addition, the man-hours required to introduce an identification algorithm for identifying the type of waste at the second facility 100 (new facility) can be reduced.

[0090] Next, the image data acquisition unit 11b acquires new image data 12b (see, for example, Figure 4) from the imaging device 6, which is an image of the inside of the waste pit 3 of the second facility 100 (step S13). The new image data 12b acquired by the image data acquisition unit 11b is stored in the storage unit 12.

[0091] Next, the type identification unit 11c takes the new image data acquired by the image data acquisition unit 11b as input and uses the identification algorithm 12a (second model) generated by the second model generation unit 11a2 to identify the type of waste stored in the waste pit 3 of the second facility 100 (step S14).

[0092] The type identification unit 11c may use the identification algorithm 12a (second model) generated by the second model generation unit 11a2, taking the new image data acquired by the image data acquisition unit 11b (see, for example, Figure 4) as input, to identify the types of waste stored in the waste pit 3 of the second facility 100, as well as the types of other objects to be identified (see, for example, Figure 6). The type identification unit 11c may also generate a map (see, for example, Figure 7) that displays the ratio of the types of waste stored in the waste pit 3 for each region, based on the waste type identification result.

[0093] Next, the plant control unit 11d controls the waste treatment plant based on the identification result from the type identification unit 11c.

[0094] Specifically, for example, the crane control unit 11d1 transmits a map showing the ratio of waste types in each region, as shown in Figure 7, as the identification result of the type identification unit 11c to the crane control device 50, which controls the crane 5 that agitates or transports the waste (step S15). Based on the map received from the crane control unit 11d1, the crane control device 50 operates the crane 5 to agitate the waste in the waste pit 3 so that the ratio of waste types is equal in all regions.

[0095] Furthermore, the combustion control unit 11d2 transmits a map to the combustion control device 20, which controls the combustion of the waste, showing the ratio of waste types for each region as the identification result of the waste type identification unit 11c (step S16). Based on the map received from the combustion control unit 11d2, the combustion control device 20 grasps the ratio of waste types that are grasped by the crane 5 and transported together from the waste pit 3 to the hopper 4, and controls the combustion of the waste according to the ratio of each type of waste that is fed together into the incinerator 1 via the hopper 4 (for example, controlling the stoker feed speed and the amount of air supplied).

[0096] Furthermore, if the type identification unit 11c identifies a worker or transport vehicle from the image data inside the waste pit 3, the fall detection unit 11e may detect the fall of the worker or transport vehicle from the platform 21 into the waste pit 3 based on the identification result of the type identification unit 11c, and transmit the identification result of the type identification unit 11c to a fall detection device (not shown).

[0097] Furthermore, if the type identification unit 11c detects an abnormal object from the image data in the waste pit 3, the foreign object input detection unit 11f may detect the abnormal object that has been put into the waste pit 3 based on the identification result of the type identification unit 11c and transmit the identification result of the type identification unit 11c to a foreign object detection device (not shown).

[0098] By the way, as shown in Figure 8, machine learning (supervised learning) requires a large amount of correct data for training. Therefore, conventional waste identification methods using identification algorithms (conventional models) require a large amount of image data and training data that must be newly prepared for each waste treatment facility.

[0099] However, when introducing identification algorithms in newly constructed facilities, the period from the start of waste acceptance to the official start of facility operation is limited, so it is necessary to collect image data, create training data, and train the algorithms within a short period of time.

[0100] Furthermore, even if image data is collected, training data is created, and training is performed within a short period, the identification accuracy of the identification algorithm will not improve for waste that appears infrequently in new facilities because there is insufficient training data.

[0101] Furthermore, since waste treatment facilities differ in the shape and dimensions of their waste pits, as well as in various environmental conditions (for example, the installation position, angle, and lighting conditions of cameras used for image acquisition), it is not possible to obtain sufficient identification accuracy by using an identification algorithm created using image data and training data from one facility at another facility.

[0102] In contrast, according to this embodiment, as shown in Figure 9, before introducing the identification algorithm to a new facility, it is possible to pre-train using a large amount of training data collected and generated at multiple facilities without time limitations, and even for waste that appears infrequently at a single facility, it is possible to train using a sufficient amount of training data. Furthermore, since training can be performed using diverse training data that takes into account differences in the shape and dimensions of waste pits at each facility, as well as differences in various environmental conditions (for example, the installation position, angle, and lighting conditions of the image acquisition camera), a robust identification algorithm 12a (first model) can be obtained. Next, the first model is further trained using training data collected and generated at the newly introduced facility to generate an identification algorithm 12a (second model) that corresponds to the waste at the newly introduced facility. Here, according to the inventors' knowledge, the amount of training data used for additional training (training data collected and generated at the newly introduced facility) should be small compared to the amount of training data used when generating the first model (training data collected and generated at multiple facilities). If the amount is small, it will be less affected by environmental conditions at the newly introduced facility (such as the installation position, angle, and lighting conditions of the image acquisition camera) and future environmental changes (such as changes in the shape and color of garbage bags and age-related soiling of the pit side walls). However, if a type of waste that does not exist in the first training data is present at the newly introduced facility, a second model capable of identifying that waste can be generated by including the training data for that waste in the training data collected and generated at the newly introduced facility. This makes it possible to obtain a model (second model) with high identification accuracy that is compatible with the newly introduced facility while maintaining a certain degree of robustness. Furthermore, compared to the waste identification method using the conventional identification algorithm (conventional model) shown in Figure 8, the man-hours required to introduce the identification algorithm 12a for identifying types of waste at the newly introduced facility can be reduced.

[0103] It is possible to make various modifications to the embodiments described above. The following describes some variations of the embodiments described above.

[0104] In the embodiment described above, as shown in Figure 2, the control unit 11 included a first model generation unit 11a1, a second model generation unit 11a2, an image data acquisition unit 11b, a type identification unit 11c, a plant control unit 11d, a fall detection unit 11e, and a foreign object input detection unit 11f. However, it is not limited to this, and some of the processing of the control unit 11 may be performed on an information processing device or cloud server other than the information processing device 10. Also, some of the storage unit 12 may be located on an information processing device or cloud server other than the information processing device 10.

[0105] For example, as shown in Figure 10, the processing of the first model generation unit 11a1 and the second model generation unit 11a2 may be executed on an external information processing device 101 (cloud server), and the identification algorithm 12a (first model and second model) may be generated. Alternatively, the processing of the type identification unit 11c may be executed on the external information processing device 101 (cloud server) using the identification algorithm 12a generated on the external information processing device 101 (cloud server), or, as shown in Figure 10, the identification algorithm 12a (second model) generated on the external information processing device 101 (cloud server) may be downloaded by the information processing device 10 installed in the second facility 100 from the external information processing device 101 (cloud server), and the processing of the type identification unit 11c may be executed using this within the information processing device 10. In this case, further learning can be performed separately on the external information processing device 101 while the second model is in operation at the second facility 100. In addition, the memory capacity of the information processing device 10 can be reduced.

[0106] After the start of operation of the identification algorithm (second model) generated by the second model generation unit 11a2, the control unit 11 may periodically monitor the identification accuracy during operation (i.e., the identification result of the type identification unit 11c) and determine whether it is necessary to revise and update the identification algorithm 12a (second model). For example, the control unit 11 uses an edge server to determine whether the identification result of the type identification unit 11c is normal or abnormal, and if an abnormality is detected, it determines whether the image data and identification result pose a problem in the operation of the incinerator 1. If it is determined that there is an operational problem, a skilled operator may re-label the image data in which an abnormality was detected by separating it by waste type, prepare new training data, and the second model generation unit 11a2 may further learn from the image data in which an abnormality was detected and the newly prepared training data to update the identification algorithm 12a (second model). This makes it possible to respond to changes in the types of waste in the second facility 100 and changes in the composition ratio of each type of waste.

[0107] The control unit 11 may, with respect to a third facility that is different from both the multiple facilities used when generating the first model and the second facility used when generating the second model, use the identification algorithm (second model) generated by the second model generation unit 11a2 as a base model, and further train the control unit 11 with third image data captured inside the waste pit of the third facility and third training data in which the waste in the image is labeled by type, thereby generating an identification algorithm (third model) corresponding to the waste of the third facility. This allows the second model to be used as a base model for further training using third training data from other facilities (third facilities), and is expected to sequentially improve the accuracy of waste type identification.

[0108] In the above-described embodiment, the combustion control unit 11d2 transmitted a map (see Figure 7) to the combustion control device 20 that displays the ratio of different types of waste stored in the waste pit 3 for each area, as the identification result of the type identification unit 11c. However, it is not limited to this, and as the identification result of the type identification unit 11c, it may also transmit a map to the combustion control device 20 that displays labels for each area that convert the ratio of different types of waste into quality information, such as "OK to input", "NG to input", "Calorie L (Low)", "M (Middle)", "H (High)", etc., as the identification result of the type identification unit 11c. The combustion control unit 11d2 may also transmit a label to the combustion control device 20 that indicates a large proportion of the impact on the combustion state (for example, "Unbroken bagged waste present", "Bottom waste present", etc.) as the identification result of the type identification unit 11c. Similarly, the crane control unit 11d1 may transmit a label to the crane control device 50 that indicates a large proportion of the impact on each piece of equipment (for example, "Pruned branches present", "Bulky crushed waste present", etc.) as the identification result of the type identification unit 11c.

[0109] In the type identification unit 11c, when generating a map that displays the ratio of different types of waste stored in the waste pit 3 for each area based on the identification result, the image data acquired by the image data acquisition unit 11b may simply be divided into areas to display the ratio of different types of waste, or the image data may be linked to the address divisions of the waste pit 3 to display the ratio of different types of waste for each address.

[0110] As a method for linking image data with crane addresses, a mark is placed on the crane 5, whose relative position to the garbage pit 3 can be measured, and the imaging device 6 captures the marked crane 5 in numerous images. Based on the numerous images in which the marked crane 5 is captured, the type identification unit 11c estimates the relative position and direction of the imaging device 6 with respect to the garbage pit 3, and then estimates the address of the pixel in the image data based on the estimated position and shooting direction of the imaging device 6. Alternatively, the imaging device 6 captures the crane 5, whose relative position to the garbage pit 3 can be measured, in numerous images, and the type identification unit 11c marks the crane 5 in the images. Based on the numerous images in which the marked crane 5 is captured, the type identification unit 11c estimates the relative position and direction of the imaging device 6 with respect to the garbage pit 3, and then estimates the address of the pixel in the image data based on the estimated position and shooting direction of the imaging device 6.

[0111] When a loading request comes from the incinerator 1, the crane control device 50 may, based on a map received from the crane control unit 11d1, select the waste that meets the loading criteria based on the ratio thresholds for each type of waste from within the waste pit 3, and then operate the crane 5 to load it into the hopper 4.

[0112] Furthermore, when the crane control device 50 sorts waste that meets the input criteria from within the waste pit 3, it may sort the waste in such a way that the difference between the ratio of waste types of the waste previously input into the hopper 4 and the sorted waste is small.

[0113] The crane control device 50 may use one or more ratio thresholds from the following as the input criteria ratio thresholds: unbroken garbage bags, paper waste, pruned branches, futons, sludge, crushed bulky waste, cardboard, burlap sacks, paper bags, bottom waste, wood chips, textile waste, clothing waste, plastic waste, animal residue, animal carcasses, kitchen waste, vegetation, soil, medical waste, incinerator ash, agricultural vinyl, PET bottles, styrofoam, meat and bone meal, agricultural products, ceramics, glass scraps, metal scraps, rubble, concrete scraps, tatami mats, bamboo, straw, and activated carbon.

[0114] Furthermore, as a method for determining the above input criteria, the crane control device 50 may determine the threshold values ​​for each type of waste that the skilled operator uses to determine whether or not to input waste by comparing the image data obtained by capturing images of the waste pit 3 in the past, with the classification and labeling of the waste by a skilled operator based on the quality of the waste shown in the image data, from the viewpoint of combustion stability and impact on equipment, and the estimation of the ratio of each type of waste using the type identification unit 11c on the image data.

[0115] Furthermore, the crane control device 50 may determine ratio thresholds for each type of waste by linking the ratio data of each type of waste actually fed into the hopper 4 with the process data of the incinerator 1, based on a map that displays the ratio of each type of waste stored in the waste pit 3 for each region, which is received from the crane control unit 11d1. Alternatively, the ratio thresholds may be dynamically changed by linking the two sets of data over time.

[0116] Furthermore, the crane control device 50 may dynamically change the ratio threshold not only based on the above process data but also on weather information. For example, if it rains on the day based on the weather information, the crane control device 50 may change the input criteria based on the ratio threshold, such as lowering the ratio threshold for unbroken garbage bags or raising the ratio threshold for crushed bulky waste.

[0117] Furthermore, the crane control device 50 may dynamically change the ratio threshold based on the day of the week information. For example, the crane control device 50 may change the input amount standard based on the ratio threshold, such as raising the ratio threshold for unbroken garbage bags on Sundays because there is less garbage in the garbage pit 3 on Sundays, in order to reduce the amount of garbage to be incinerated.

[0118] Furthermore, the crane control device 50 may dynamically change the ratio threshold based on the furnace operation plan values ​​of the waste treatment facility 100. For example, if the evaporation rate has fallen below the current evaporation rate setting, the crane control device 50 may change the input criteria based on the ratio threshold, such as lowering the ratio threshold for unbroken garbage bags or raising the ratio threshold for crushed bulky waste.

[0119] When a request for incineration is received from the incinerator 1, the crane control device 50 may, if it receives a map from the crane control 11d1 and there is no waste that meets the incineration criteria based on the ratio thresholds for each type of waste, operate the crane 5 to inflate waste that is close to the incineration criteria into the hopper 4, or it may agitate the waste that is close to the incineration criteria to create waste that meets the incineration criteria.

[0120] The crane control device 50 may operate the crane 5 to pile up only the waste that meets the input criteria based on the ratio thresholds for each type of waste, as shown in the map received from the crane control 11d1, at a specific location in the waste pit 3. In this way, waste that meets the input criteria can be accumulated in the waste pit 3.

[0121] The crane control device 50, using a map received from the crane control 11d1, detects waste that affects the combustion state (e.g., sludge) or waste that causes problems for each piece of equipment (e.g., pruned branches) present in the waste pit 3 based on ratio thresholds for each type of waste. It may then operate the crane 5 to store the waste in a specific location in the waste pit 3 or scatter it in a specific location.

[0122] The crane control device 50 may operate the crane 5 to agitate the waste if, in the map received from the crane control 11d1, there is waste in the waste pit 3 that does not meet the agitation criteria based on the ratio threshold for each type of waste. The agitation criteria may be the same as or different from the input criteria.

[0123] Furthermore, the crane control device 50 may dynamically change the above stirring criteria using one or more of the following: process data of the incinerator 1, weather information, day of the week information, waste transporter information, waste transport volume (total volume and amount transported by waste type) information, waste transport speed, waste pit level (overall, specific area) information, crane operating status (2 cranes can be operated, only 1 crane is operating, 1 crane is currently operating, 2 cranes are currently operating) information, and waste collection vehicle collection route / collection area information.

[0124] The crane control device 50 may determine the overall mixing status of the waste pit 3 from the ratio of different waste types in each area in the map received from the crane control 11d1, determine whether it is necessary to operate two cranes 5, operate crane 5, start operation of the second crane, or retract the second crane.

[0125] Furthermore, in the above example, the crane control device 50 operated the crane 5, but a crane operation determination device (not shown) may be provided upstream of the crane control device 50, which may determine the operation of the crane 5 and send a command for the operation to the crane control device 50, and the crane control device 50, upon receiving the command, operates the crane 5 based on the received command. The crane operation determination device sends and receives information with the information processing device 10 and with the crane control device 50. Alternatively, the crane operation determination device may be part of the information processing device 10, that is, the information processing device 10 may include the crane operation determination device.

[0126] The crane operation decision device receives a loading request signal from the crane control device 50 when a loading request comes from the incinerator 1. Based on the map received from the crane control unit 11d1, it selects the waste in the waste pit 3 that meets the loading criteria based on the ratio thresholds for each type of waste, and sends a command to the crane control device 50 to load the waste into the hopper 4. The crane control device 50 may then operate the crane 5 based on the received command. Alternatively, when the crane operation decision device selects the waste in the waste pit that meets the loading criteria, it may select the waste in such a way that the difference between the ratio of each type of waste and the waste previously loaded into the hopper 4 is small.

[0127] When a request for loading is received from the incinerator 1, the crane operation decision device receives a loading request signal from the crane control device 50. If, in the map received from the crane control 11d1, there is no waste that meets the loading criteria based on the ratio thresholds for each type of waste, the device sends a command to the crane control device 50 to load waste that is close to the loading criteria into the hopper 4, or to agitate the waste that is close to the loading criteria to create waste that meets the loading criteria, and the crane control device 50 may then operate the crane 5.

[0128] The crane operation decision device receives a map from the crane control 11d1 and sends a command to the crane control device 50 to pile up only the waste that meets the input criteria based on the ratio threshold of waste types at a specific location in the waste pit 3. The crane control device 50 may then operate the crane 5. In this way, waste that meets the input criteria can be accumulated in the waste pit 3.

[0129] The crane operation decision device, using a map received from the crane control 11d1, detects waste that may affect the combustion state (e.g., sludge) or waste that may cause problems with each piece of equipment (e.g., pruned branches) present in the waste pit 3 based on ratio thresholds for each type of waste. It then sends a command to the crane control device 50 to store the waste in a specific location within the waste pit 3 or to scatter it in a specific location, and the crane control device 50 may then operate the crane 5.

[0130] The crane operation decision device, when it receives a map from the crane control 11d1 and finds that there is waste in the waste pit 3 that does not meet the agitation criteria based on the ratio thresholds for each type of waste, sends a command to the crane control device 50 to agitate the waste, and the crane control device 50 may operate the crane 5. The agitation criteria may be the same as or different from the input criteria.

[0131] The crane operation decision device, based on the map received from the crane control 11d1, determines the overall mixing status of the waste pit 3 from the ratio of different waste types in each area, determines whether it is necessary to operate two cranes 5, and sends a command to the crane control device 50. The crane control device 50 may then operate crane 5 to start operation of the second crane, or to retract the second crane.

[0132] In the above-described embodiment, an example was explained in which the information processing device 10 for identifying the type of waste is used in the waste pit 3 of the waste treatment facility 100. However, the location where the information processing device 10 can be used is not limited to the waste pit 3 of the waste treatment facility 100, as long as it is a waste storage area where waste is stored. For example, the information processing device 10 may be used at the receiving area of ​​a recycling facility.

[0133] Although embodiments and modifications of the present invention have been described above by example, the scope of the present invention is not limited thereto, and it is possible to modify and transform it according to the purpose within the scope described in the claims. Furthermore, each embodiment and modification can be appropriately combined as long as the processing content is not contradictory.

[0134] Furthermore, although the information processing device 10 according to this embodiment may be composed of one or more computers, the program for implementing the information processing device 10 on one or more computers and the recording medium on which the program is stored non-temporarily are also subject to protection in this case. [Explanation of symbols]

[0135] 1 Incinerator 2 Combustion device 3. Garbage pit 4 Hoppers 5 Cranes 6. Imaging device 10 Information Processing Devices 11 Control Unit 11a1 First Model Generation Unit 11a2 Second Model Generation Unit 11b Image data acquisition unit 11c Type identification part 11d Plant Control Unit 11e Fall detection unit 11f Foreign object detection unit 12 Storage section 12a Identification Algorithm 12b Image data 13 Communications Department 14 Display section 20 Combustion control device 21 Platforms 22 Transport Vehicles 50 Crane control device 100 Waste Treatment Facilities 101 External information processing device

Claims

1. The process involves generating a first model, which is a waste type identification algorithm, by machine learning using first image data, which consists of images taken inside waste pits at multiple facilities where waste is stored, and first training data, which consists of waste labeled by type within those images. An information processing method comprising the step of generating a second model, which is an identification algorithm corresponding to the waste of the second facility, by further training the generated first model with second image data taken inside a waste pit of a second facility different from the plurality of facilities, and second training data in which the waste in the image is labeled by type, the first model being generated, In cases where a type of waste not present in the first training data exists at the second facility, the second training data includes training data labeled with the type of waste, thereby generating the second model capable of identifying the waste. Information processing methods.

2. The process involves generating a first model, which is a waste type identification algorithm, by machine learning using first image data, which consists of images taken inside waste pits at multiple facilities where waste is stored, and first training data, which consists of waste labeled by type within those images. An information processing method comprising the step of generating a second model, which is an identification algorithm corresponding to the waste of the second facility, by further training the generated first model with second image data taken inside a waste pit of a second facility different from the plurality of facilities, and second training data in which the waste in the image is labeled by type, the first model being generated, In cases where there are types of waste present in the first training data that do not exist in the second facility, the second training data is characterized in that the types of waste to be labeled are not included in the second training data. Information processing methods.

3. The amount of the second training data relative to the first training data is 30% or less. The information processing method according to claim 1 or 2.

4. The amount of the second training data relative to the first training data is 15% or less. The information processing method according to claim 3.

5. The amount of the second training data relative to the first training data is 5% or less. The information processing method according to claim 4.

6. The aforementioned discrimination algorithm includes one or more of the following: linear regression, Boltzmann machines, neural networks, support vector machines, Bayesian networks, sparse regression, decision trees, statistical estimation using random forests, reinforcement learning, and deep learning. The information processing method according to any one of claims 1 to 5.

7. The second training data includes labeled images of the entire contents of a waste pit in the second facility. The information processing method according to any one of claims 1 to 6.

8. The second training data includes a portion of an image taken inside the waste pit of the second facility, with only that portion labeled. The information processing method according to any one of claims 1 to 6.

9. The second image data used in the step of generating the second model is obtained by inputting image data captured inside the waste pit of the second facility into the first model to identify the type of waste, and then selecting from that image data image data that has an identification accuracy lower than a predetermined first criterion. The information processing method according to any one of claims 1 to 8.

10. In the step of generating the first model, information on the shooting conditions and / or shooting environment at each facility is learned together. The information processing method according to any one of claims 1 to 9.

11. The first image data is obtained by taking images of the inside of the waste pit of each facility and applying correction to at least one of the following: hue, brightness, and saturation, using an image of a common image correction color chart taken across the multiple facilities as a reference. The information processing method according to any one of claims 1 to 10.

12. The first image data was captured inside the waste pit of each facility, along with a common color chart for image correction used across the multiple facilities. The information processing method according to any one of claims 1 to 10.

13. The second image data includes a composite image of the waste in the waste pits of the multiple facilities, or the waste in the waste pits of other facilities different from both the multiple facilities and the second facility, onto a rendering image of the side wall of the waste pit and / or a crane used for agitating or transporting the waste, which was created based on the three-dimensional design data of the second facility. The information processing method according to any one of claims 1 to 12.

14. After the second model is put into operation, the identification accuracy during operation is periodically monitored, and if the identification accuracy falls below a predetermined second standard, the second model is updated by additionally training it with the image data of the waste at that time and training data in which the waste in the image is labeled by type. The information processing method according to any one of claims 1 to 13.

15. The process further includes the step of generating a third model, which is an identification algorithm corresponding to the waste of the third facility, by further training the generated second model with third image data, which is image data of the inside of a waste pit of a third facility different from both the plurality of facilities and the second facility, and third training data, which is data in which the waste in the image is labeled by type. The information processing method according to any one of claims 1 to 14.

16. A first model generation unit generates a first model, which is a waste type identification algorithm, by machine learning using first image data, which is image data taken inside the waste pits of multiple facilities where waste is stored, and first training data, which is data in which the waste in the image is labeled by type. An information processing device comprising a second model generation unit that generates a second model, which is an identification algorithm corresponding to the waste of the second facility, by additionally training the generated first model with second image data captured from inside a waste pit of a second facility different from the plurality of facilities, and second training data in which the waste in the image is labeled by type, the first model being generated, When a type of waste not present in the first training data exists at the second facility, the second model generation unit generates a second model capable of identifying the waste by including training data labeled with the type of waste in the second training data. Information processing device.

17. On the computer, The process involves generating a first model, which is a waste type identification algorithm, by machine learning using first image data, which consists of images taken inside waste pits at multiple facilities where waste is stored, and first training data, which consists of waste labeled by type within those images. An information processing program for performing the steps of generating a second model, which is an identification algorithm corresponding to the waste of the second facility, by further training the generated first model with second image data taken inside a waste pit of a second facility different from the plurality of facilities, and second training data in which the waste in the image is labeled by type, the first model being generated, When a type of waste not present in the first training data exists at the second facility, the second training data includes training data labeled with the type of waste, thereby enabling the generation of the second model capable of identifying the waste. Information processing program.

18. The first model is a waste type identification algorithm generated by machine learning using first image data of waste pits in multiple facilities where waste is stored, and first training data in which the waste in the images is labeled by type. The second model is an identification algorithm corresponding to the waste in the second facility, generated by further training using second image data of waste pits in a second facility different from the multiple facilities, and second training data in which the waste in the images is labeled by type. The second model is then used to identify the type of waste stored in the waste pit of the second facility, taking new image data of the waste pit of the second facility as input. An information processing method including, In cases where a type of waste not present in the first training data exists in the second facility, the second model is characterized by being generated by adding training data labeled with the type of waste to the second training data, thereby enabling the identification of the waste. Information processing methods.

19. A type identification unit identifies the type of waste stored in a waste pit by taking new image data taken from the waste pit of the second facility as input. This type identification unit uses a second model, which is an identification algorithm for waste corresponding to the waste of the second facility, generated by machine learning a first model, an identification algorithm for waste type identification algorithm, which is generated by further training a second model, an identification algorithm for waste of the second facility, which is generated by further training a second model, an identification algorithm for waste of the second facility, which is a type of waste identification algorithm generated by further training a second model, an identification algorithm for waste of the second facility, which is a type of waste identification algorithm generated by further training a second model, an identification algorithm for waste of the second facility, which is a type of waste identification algorithm generated by further training a second model, an identification algorithm for waste of the second facility, which is a type of waste identification algorithm generated by further training a second model, an identification algorithm for waste of the second facility, which is a type of waste identification algorithm, which is a type of waste identification algorithm generated by further training a second model, an identification algorithm for waste of the second facility, which is a type of waste identification algorithm, which is a type of waste for which waste corresponds to the waste of the second facility, which is a type of waste identification algorithm, which An information processing device equipped with, In cases where a type of waste not present in the first training data exists in the second facility, the second model is characterized by being generated by adding training data labeled with the type of waste to the second training data, thereby enabling the identification of the waste. Information processing device.

20. On the computer, The first model is a waste type identification algorithm generated by machine learning using first image data of waste pits in multiple facilities where waste is stored, and first training data in which the waste in the images is labeled by type. The second model is an identification algorithm corresponding to the waste in the second facility, generated by further training using second image data of waste pits in a second facility different from the multiple facilities, and second training data in which the waste in the images is labeled by type. The second model is then used to identify the type of waste stored in the waste pit of the second facility, taking new image data of the waste pit of the second facility as input. An information processing program for executing, In cases where a type of waste not present in the first training data exists in the second facility, the second model is characterized by being generated by adding training data labeled with the type of waste to the second training data, thereby enabling the identification of the waste. Information processing program.

Images