Method for intelligent control of waste collection in an automated waste collection plant

Abstract

The present invention relates to a method for the intelligent control of waste collection in an automated waste collection plant. This method comprises storing operational information from the plant and an energy value corresponding to the energy used in the emptying cycles; calculating an energy consumption model using a first algorithm; and training a virtual agent using a second algorithm that evaluates the energy consumption of a series of plant actions in response to different waste occupancy levels at the inlets. The virtual agent receives, in real time, information regarding the load status of the inlets of a real plant and establishes decision-making policies for that real plant based on the information received and the actions deemed optimal during training. The real plant is then operated according to the established decision-making policies.

Description

FIELD OF THE INVENTION

[0001] The present invention generally relates to waste collection methods and systems. In particular, the invention relates to a computer-implemented method for the intelligent / smart control of waste collection in an automated waste collection plant.BACKGROUND OF THE INVENTION

[0002] In general, an automated waste collection plant (or system / facility) normally uses air suction by means of a vacuum system in a closed network of underground pipes, covering an area of a few square kilometers, for conveying the waste from delivery points, which comprise inlets, distributed for example in one part or all over a city, to a central collection point, reducing greenhouse gas emissions and the drawbacks of conventional methods (smells, noise, etc.), in addition to allowing the waste to be better reused and recycled.

[0003] From a plant operation viewpoint, automated waste collection plants today are generally rather inefficient in terms of energy and on an economic level mainly due to the fact that waste collection or removal is neither adapted to plant occupation levels nor to the energy cost during collection.

[0004] Likewise, to enable modeling and controlling an automated waste collection plant correctly, precise knowledge of the different parts of said plant is required.

[0005] The emergence and evolution of automatic learning-based artificial intelligence techniques has made it possible to approach such modeling and control.

[0006] For example, patent application EP2666737A1, belonging to the same inventors as the present invention, provides a method for the removal of waste in an automated waste collection plant where automatic learning techniques or algorithms are used for the planning operations of plants. The objective is to achieve optimal operation plans, at any time, based on the current state of the different parts of the plants and on previous experiences, which minimize energy consumption subject to pre-established service quality standards. This method uses machine learning techniques and linear programming algorithms for the planning operations of the automated waste collection plant; however this previous patent application is completely silent on the different models computed by the present invention and in the provision and training of a virtual agent to evaluate the energy consumption of a series of actions of a waste collection plant and to establish emptying control decision-making policies for a real waste collection plant. Hence, although this previous method has been proven to be efficient, there is still room for reducing energy consumption, and also the cost, involved with the operation of these plants.

[0007] KR20190113015A relates to an artificial intelligent automatic collection system based on database capable of minimizing maintenance costs. Information such as detection of a level sensor, a number of inlet door openings, and a weighing of a waste weight is collected in a garbage inlet, and stored in database, and is analyzed so as to selectively performing automatic collection only from an inlet which meets a certain conditions. This system neither includes nor uses the cited virtual agent of present invention.

[0008] Therefore, new methods for the intelligent / smart control of waste collection in an automated waste collection plant are required.DESCRIPTION OF THE INVENTION

[0009] To this end, embodiments of the present invention provide a computer-implemented method for the intelligent control of waste collection in an automated waste collection plant. The proposed method comprises storing, in a memory or database, operating information of a plurality of real automated waste collection plants, wherein the operating information at least includes, for each of a number of the emptying cycles of each real plant: the number of inlets (i.e. the containers / boxes were the waste (i.e. rubbish / garbage) is put / stored) of the plant being emptied, the distance from the emptied inlets to a collection center (or central collection point), the type of waste, the volume of the waste, and the time instant in which the emptying cycle takes place. Additionally, an energy value corresponding to the energy used in the corresponding emptying cycle is also stored in the memory or database.

[0010] The method also comprises generating / computing, by a processing unit, an energy usage model (i.e. a model of the energy consumption of the emptying cycles of one or more inlets) of at least one specific automated waste collection plant. To do so, the processing unit runs / executes a first artificial intelligence algorithm, comprising a first neural network, that, for each emptying cycle of the specific automated waste collection plant: a) computes a predicted energy value using as inputs the operating information stored in the memory or database, b) compares the computed predicted energy value with the energy value stored in the memory or database; and c) adjusts the first neural network (i.e. adjusts its weight values / parameters) if the comparison is different; where steps a) to c) are repeated until a comparation result is below or equal a given error threshold.

[0011] The method also comprises training, by a processing unit, a virtual agent by means of running a second artificial intelligence algorithm comprising a second neural network. The second neural network evaluates the energy consumption of a series of actions of the specific automated waste collection plant (or of any other automated waste collection plant), in response to a plurality of waste loading / occupation situations in the inlets, by means of: i) selecting, given a loading state of the specific / other automated waste collection plant considered, a given action to perform; ii) evaluating a cost of said given action using the generated energy usage model; and iii) minimizing the cost by means of adjusting the second neural network (i.e. adjust the weight values / parameters thereof). Steps i) to iii) are repeated until the cost is below or equal a given threshold or for a limited number of repetitions.

[0012] The method also comprises receiving, by the virtual agent, in real time, information relating to a loading / capacity state of the inlets of a real automated waste collection plant, and the virtual agent establishing one or more emptying control decision-making policies of the real automated waste collection plant based on the received information and on actions considered optimal during the training. Lastly, the real automated waste collection plant is operated according to the mentioned emptying control decision-making policy / policies established by the virtual agent.

[0013] In one embodiment, the method further calculates a time duration model of the time each emptying cycle takes, particularly by means of running a third artificial intelligence algorithm, comprising a third neural network, on at least part of the operating information stored in the memory or database.

[0014] The first and third neural networks particularly comprise a deep learning-based neural network. Likewise, the second neural network comprises a deep reinforcement learning-based neural network.

[0015] In some particular embodiments, the different neural networks comprise a Deep Feed Forward (DFF). The features of the invention can nevertheless be implemented via other neural networks.

[0016] In one embodiment, the operating step is performed by a controller device operatively connected to each of the inlets of the real waste collection plant, after having received the control decision from the virtual agent.

[0017] In one embodiment, prior to the step of operating the real waste collection plant according to the mentioned emptying control decision-making policy / policies established by the virtual agent, it is verified from information relating to the loading state of the inlets of the real waste collection plant if the emptying control decision-making policy / policies adopted is / are determining factor(s) of a real energy consumption optimization.

[0018] According to the proposed method, the mentioned emptying control decision-making policy / policies may comprise emptying one or more of the inlets of the real waste collection plant; activating or deactivating vacuum-based emptying systems for emptying the inlets; and / or increasing or reducing the speed of vacuum-based emptying systems for emptying the inlets, among others.

[0019] The specific waste collection plant may comprise a single waste collection plant, or alternatively a virtual waste collection plant which compiles operating information of a plurality of real waste collection plants.

[0020] Other embodiments of the invention disclosed herein also include computer program products for performing the steps and operations performed by the proposed method. More particularly, a computer program product is an embodiment having a computer-readable medium including code instructions encoded therein which, when run in at least one processor of the computer system, cause the processor to perform the operations indicated herein as embodiments of the invention.BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The foregoing and other features and advantages will be better understood from the following merely illustrative and non-limiting detailed description of embodiments in reference to the attached drawings, in which: Fig. 1 is a flowchart illustrating a method for the intelligent control of waste collection in an automated waste collection plant, according to one embodiment of the present invention. Fig. 2 schematically illustrates the architecture of a system for the intelligent control of waste collection, according to one embodiment of the present invention. Fig. 3 schematically illustrates the architecture used for operating the virtual agent. DETAILED DESCRIPTION OF THE INVENTION

[0022] With reference to Figs. 1 and 2, an embodiment of the proposed method is shown therein. According to this embodiment, in step 101 (or phase 0), the method comprises generating an energy usage model (or first prediction model) of a specific automated waste collection plant (or simply collection plant), for example a specific waste collection plant or a fictitious waste collection plant programmed / configured for acquiring operating information of a series of real waste collection plants.

[0023] Particularly, the energy usage model is generated by a processing unit that predicts the energy consumption of the emptying cycles of the inlets (i.e., high-speed startup operation, plus the emptying of a series of inlets, plus transport to the central collection point) by means of implementing / executing a first artificial intelligence algorithm, for example, a deep learning-based neural network, using at least part of the operating information of a plurality of real collection plants that is stored in a memory or database.

[0024] The operating information comprises real / actual data of each of the emptying cycles of each real collection plant, for instance: the number of inlets of the collection plant being emptied, the distance from the emptied inlets to a collection center, the type of waste, the volume of the waste, the time instant in which the emptying cycle takes place, etc.

[0025] The memory or database also stores an energy value corresponding to the energy used in the emptying cycle.

[0026] Particularly, the first deep learning-based neural network, for each emptying cycle of the collection plant considered a) computes a predicted energy value using as input (part of or all of) the operating information stored in the memory or database; b) compares the computed predicted value with the stored energy value; and c) adjusts the first deep learning-based neural network until the comparison result is below or equal a given error threshold.

[0027] Continuing with the explanation of Fig. 1, once the energy usage model capable of indicating the energy consumption of an emptying cycle has been generated, in step 102 (or phase 1), the method comprises training a virtual agent (i.e. software) by means of implementing a second artificial intelligence algorithm. The second algorithm particularly comprises a neural network based on reinforced learning or deep reinforcement learning-based neural network that is trained during a specified number of episodes / repetitions or until a cost reaches a specified threshold. The trained neural network will be used as the policy agent for the following stages, as explained above.

[0028] In step 103 (or phase 2), the previously trained virtual agent receives, in real time, information relating to a loading / capacity state of the inlets of a real collection plant (i.e. a collection plant to be controlled) and establishes one or more emptying control decision-making policies of said real plant taking into consideration the received information and actions considered optimal during training. In one embodiment, information relating to the loading state of the inlets of the real collection plant is received, particularly, via a controller device 10, such as a PLC, installed at the real collection plant and operatively connected to each of the inlets of the real collection plant.

[0029] Lastly, in step 104 (or phase 3), the method comprises operating the real collection plant according to the mentioned one or more emptying control decision-making policies established by the virtual agent. In particular, this final operating step is also performed by the controller device 10. In any event, it should be noted that in other embodiments, the operating step could be performed by a computer or PC, among other computing systems.

[0030] Likewise, particularly, step 104 is performed once a sufficient time interval in which the virtual agent has not established any incorrect control decision-making policy has passed.

[0031] Referring now to Fig. 3, therein an embodiment of how the virtual agent is operated is illustrated. In this embodiment, the training that the virtual agent undergoes is particularly done by considering 1) a simulated plant, which can be a generic non-existing collection plant or a specific existing collection plant, and 2) a sequence of loading / occupation states of such collection plant (i.e., a temporal sequence of user waste drops to the inlets of the simulated plant). Then, using such a simulated plant, the second artificial intelligence algorithm during a given number of episodes / repetitions N or till the simulation reward / cost of the actions reaches a decided threshold first restarts the simulation and puts to zero an actual reward / cost. Then, for a given number of time slots T: Gets the loading state of the simulated plant (taking into consideration a previous loading state and the user waste drops from the simulated scenario); Selects the most adequate action (i.e. the optimal) from the policy, given by virtual agent, or a random action with probability E; simulates the effect of the selected action on the simulated plant (i.e. chaining the state of such plant);

[0032] The cost of the action is the economic cost of the energy usage of that action as given by the energy usage model from step 101 and can be altered by other programmed factors (e.g. penalties for not emptying the inlets, etc.) or by solar energy generation availability or energy price; Adds the cost to the simulation cost; Saves the state, action, and cost to be used later. (If used) the probability E of choosing random actions is decreased.

[0033] Finally, particularly periodically, all or some of the saved state, action, cost tuples are used as a policy.

[0034] In the present invention, the controller device 10 and the virtual agent may operate in parallel. In other words, while the controller device 10 operates the real collection plant, the virtual agent simultaneously receives information about the state of the inlets and makes the corresponding decisions. The virtual agent thereby improves continuously throughout the operation time of the real collection plant.

[0035] In one embodiment, these decisions are not directly taken into consideration by the controller device 10, but rather said decisions are previously sent to a monitoring and telemetry system 20 for validation. In other examples, the controller device 10 receives the decisions directly from the virtual agent. In any case, the virtual agent may act as a supervisor to avoid mistaken operating actions in the real collection plant.

[0036] In some embodiments, the calculations also comprise implementing a third artificial intelligence algorithm, for example, a deep learning-based neural network, on at least part of the operating information stored in the memory or database for calculating a time duration model (or second prediction model) of the time each emptying cycle takes. Additional variables, such as the price of the energy (based on the time), temporal data (day of the week, etc.), time prediction if solar energy is used in the collection plant, etc., which allow even further optimizing the simulated model, are thereby taken into account.

[0037] Therefore, in the present invention the virtual agent is configured for observing one or more real collection plants to be controlled, as well as their environment (for example, the cost of energy, the time of the day on which waste collection takes place, etc.), such that the virtual agent can establish / provide the different decision-making control policies and perform actions that affect the environment of the mentioned collection plant(s), for example establishing emptying sequences, activating or deactivating vacuum-based emptying systems for emptying the inlets, increasing or reducing the speed of vacuum-based emptying systems, etc. The virtual agent is configured for performing the preceding actions hundreds or thousands of times taking different scenarios into consideration.

[0038] The proposed invention can be implemented in hardware, software, firmware, or any combination thereof. If it is implemented in software, the functions can be stored in or be encoded as one or more code instructions in a computer-readable medium.

[0039] The scope of the present invention is defined in the attached claims.

Claims

1. A method for the intelligent control of waste collection in an automated waste collection plant, comprising: storing operating information of a plurality of real automated waste collection plants in a memory or database, wherein the operating information includes, for each of a number of the emptying cycles of each plant: number of inlets of the plant being emptied, distance from the emptied inlets to a collection center, type of waste, volume of the waste, and time instant in which the emptying cycle takes place, and further storing an energy value corresponding to the energy used in the corresponding emptying cycle in the memory or database; generating, by a processing unit, an energy usage model of a specific automated waste collection plant by running a first artificial intelligence algorithm comprising a first neural network, the first neural network, for each emptying cycle of the specific automated waste collection plant, comprising: a) computing a predicted energy value using as inputs the operating information stored in the memory or database, b) comparing the computed predicted energy value with the energy value stored in the memory or database; and c) adjusting the weighted associations of the first neural network if the comparison provides a different result; and steps a) to c) being repeated until a comparation result is below or equal a given error threshold; training, by a processing unit, a virtual agent by means of running a second artificial intelligence algorithm comprising a second neural network, the second neural network evaluating the energy consumption of a series of actions of the specific automated waste collection plant or of another automated waste collection plant, in response to a plurality of waste loading situations in the inlets, by means of: i) selecting, given a loading state of the specific / another automated waste collection plant, a given action to perform; ii) evaluating a cost of said given action using the generated energy usage model; iii) minimizing the cost by means of adjusting the weighted associations of the second neural network; steps i) to iii) being repeated until the cost is below or equal a given threshold or for a limited number of repetitions; receiving, by the virtual agent, in real time, information relating to a loading state of the inlets of a real automated waste collection plant, and establishing, by the virtual agent, one or more emptying control decision-making policies of the real automated waste collection plant based on the received information and on actions considered optimal during the training of the virtual agent; and operating the real automated waste collection plant according to said one or more emptying control decision-making policies established by the virtual agent.

2. The method according to claim 1, further comprising calculating a time duration model of the time each emptying cycle takes by means of running a third artificial intelligence algorithm, comprising a third neural network, on at least part of the operating information stored in the memory or database.

3. The method according to claim 1, wherein the first neural network comprises a deep learning-based neural network.

4. The method according to claim 2, wherein the third neural network comprises a deep learning-based neural network.

5. The method according to any one of the preceding claims, wherein the second neural network comprises a deep reinforcement learning-based neural network.

6. The method according to any one of the preceding claims, wherein the operating step is performed by a controller device, operatively connected to each of the inlets of the real automated waste collection plant, after having received the control decision from the virtual agent.

7. The method according to any one of the preceding claims, comprising, prior to the step of operating the real automated waste collection plant according to said one or more emptying control decision-making policies established by the virtual agent, verifying, by a processing unit, from information relating to the loading state of the inlets of the real automated waste collection plant if the emptying control decision-making policies adopted are determining factors of a real energy consumption optimization.

8. The method according to any one of the preceding claims, wherein said one or more emptying control decision-making policies comprise at least one of: emptying one or more of the inlets of the real automated waste collection plant; activating or deactivating vacuum-based emptying systems for emptying the inlets; and / or increasing or reducing the speed of vacuum-based emptying systems for emptying the inlets.

9. The method according to any one of the preceding claims, wherein the specific automated waste collection plant comprises a virtual automated waste collection plant which compiles operating information of a plurality of real waste collection plants.

10. A computer program product including code instructions which, when implemented in a computing system, implement a method according to any one of claims 1 to 9.

Images