Waste source identification

Abstract

Method of monitoring waste, comprising: obtaining waste images 502; inputting the images to a material recognition model 504 trained to output position 506 and classification 508 of objects; inputting the object location into a source determination model 510 trained to determine an waste object source score 514 i.e. likelihood of a waste object’s origin (e.g. household, business, type, class, specification, source of manufacture, method of manufacture, entity and / or brand). The material recognition model 502, (300,Fig.3) may comprise: a feature extraction model (304,Fig.3) that outputs a feature map (304, Fig.3) which is input to an object detector (308,Fig.3) and an object classifier (310,Fig.3). The source determination model 510, (400,Fig.4) may comprise: an object feature extraction model (408,Fig.4) that outputs an object level feature map which is then input into source classification layers (410,Fig.4) that determine an object’s source score (414,Fig.4) and feature layers that determine object feature vectors (416,Fig.4). Waste identifier database 526 may store object source scores and object vectors and further be queried by a user and query result 528 output. Duplication stage 512 may avoid storing the same object twice by assigning a unique identification 516 to objects; results of images of the same object being averaged 520 before storage.

Description

Technical Field

[0001] The invention relates to computer-implemented methods and systems for processing waste at one or more material processing facilities. Background

[0002] Waste management refers to the process of collecting, transporting, treating, and disposing of waste materials in an environmentally responsible and efficient manner. The goal of waste management is to reduce the negative impact of waste on human health and the environment. Historically, waste management has been a significant challenge for communities around the world, as increasing populations and economic development have led to a corresponding increase in the amount of waste generated. As a result, waste management has become an increasingly important issue in the modern era, with governments and organizations seeking to develop sustainable and effective waste management practices.

[0003] One of the key challenges in waste management is identifying the source and / or the type of waste. This can provide useful information to businesses, governments, and individuals. Waste can originate from a wide range of sources, including households, businesses, construction sites, and industrial processes. Identifying the source of waste is essential for developing effective waste management strategies, as different sources of waste require different methods of disposal, processing, or treatment. Summary

[0004] There is described herein a computer-implemented method for monitoring waste at one or more material processing facilities each comprising one or more waste monitoring units. The method comprises receiving waste at a material processing facility, the waste comprising one or more objects. The method further comprises obtaining, by a waste monitoring unit of the material processing facility, one or more images of the waste. The method further comprises providing data representing the one or more images as input to a first model configured to output, for at least one of the one or more objects depicted in the one or more images, data indicating the at least one object. The method further comprises outputting an object source score for each of the at least one object, comprising providing the data indicating the at least one object as input to a source determination model configured to output a source determination model output indicating one or more corresponding object source scores, each object source score indicative of a likelihood that a source of one of the at least one objects depicted in the one or more images can be determined.

[0005] Advantageously, by generating the object source scores each indicative of the likelihood that an object in the waste can be determined, the source of an object in the waste may be automatically determined. In addition, waste production may be monitored and / or identified using the one or more object source scores, thereby providing useful information for controlling waste management. In some embodiments, the data representing the one or more images may be the one or more images or may be one or more further images obtained by processing the one or more images, for example, by using any appropriate image processing techniques.

[0006] The method may further comprise determining, based upon the data indicating the at least one object, one or more object feature vectors indicative of the at least one object depicted in the one or more images. By generating the object feature vectors indicative of the at least one object depicted in the one or more images, one or more representations of the at least one object may be generated. For example, the object feature vectors may be used to automatically classify waste based upon their vector representation. In some embodiments, the object feature vectors may be determined by the source determination model. In some embodiments, the object feature vector may comprise one or more numerical values.

[0007] The method may further comprise querying, based upon the one or more object source scores and / or the one or more object feature vectors, a waste identifier database to determine one or more corresponding waste unit classifications for each of the at least one object, the waste unit classifications each corresponding to one of one or more waste unit classes in the waste identifier database. By querying the waste identifier database using the one or more object feature vectors, waste unit classification may be efficiently and automatically provided for. In addition, because the waste identifier database may be queried by one or more waste monitoring units, waste unit classification may be provided for in a distributed manner, thereby improving the overall efficiency and accuracy of waste unit classification. In some embodiments, the waste identifier database may be implemented on one or more servers. In some embodiments, the waste identifier database may comprise example data and corresponding classification data. In some embodiments, the example data comprises one or more example object source scores and / or one or more example object feature vectors, and the classification data comprises one or more waste unit classifications corresponding to the one or more example object source scores and / or one or more example object feature vectors. In some embodiments, the waste identifier database may comprise one or more trained machine learning models. In some embodiments, the waste identifier database may be configured to transmit the one or more trained machine learning models to the one or more waste monitoring units described herein in order to update their respective models. In such a way, the waste identifier database may improve waste unit classification by providing improved machine learning models to the one or more waste monitoring units.

[0008] In some embodiments, at least one of the at least one waste unit classification corresponds to one or more stock keeping units. Advantageously, stock keeping units corresponding to waste unit classifications may enable waste to be more easily tracked. For example, waste production of particular products may be catalogued and monitored according to the waste unit classifications and their corresponding stock keeping units. In some embodiments, each of the at least one waste unit classification may correspond to a different stock keeping unit.

[0009] In some embodiments, the first model is a material recognition model configured to output one or more object material classifications. The material recognition model may provide one or more object material classifications in order to recognize that certain objects comprise certain materials. In such a way, the identification of objects comprising such materials is provided for, which contributes to an overall improvement in waste unit classification where such identification is required. In addition, the object material classifications may be used to track the same object depicted in multiple images, thereby providing for improved tracking, deduplication, hence an improvement in overall waste unit classification. In some embodiments, the material recognition model may be a trained machine learning model. In some embodiments, the material recognition model may be a trained neural network. In some embodiments, the material recognition model may have been trained using ground truth training data.

[0010] In some embodiments, the first model comprises a feature extraction model configured to receive as input the data representing the one or more images and to output an image-level feature map comprising data indicative of one or more features corresponding to the one or more images, an object detector configured to receive as input the feature map and to output the data indicating the at least one object in the one or more images, and an object classifier configured to receive as input the feature map and to output one or more object classifications.

[0011] Advantageously, where the first model may comprise a feature extraction model, an object detector, and an object classifier, features of one or more objects may be extracted, localized, and classified, all using a single model. This improves overall waste unit classification by providing a more compact waste classification pipeline with fewer processing steps. In some embodiments, the feature extraction model may be a locality preserving network. In some embodiments, the feature extraction model may be a backbone network (e.g. a feature extracting portion of a trained neural network). In some embodiments the feature extraction model may be a trained machine learning model and / or a trained neural network. In some embodiments, the feature extraction model may be one or more layers of a trained machine learning model. In some embodiments, the object detector may and / or the object classifier may be a machine learning model. In some embodiments, the object detector may and / or the object classifier may be one or more layers of a machine learning model. In some embodiments, the object detector may and / or the object classifier may be a trained neural network and / or one or more layers of a trained neural network.

[0012] In some embodiments, the source determination model comprises an object feature extraction model configured to receive as input the data indicating the at least one object in the one or more images and to output an object-level feature map, one or more source classification layers, the one or more source classification layers connected to the object feature extraction model to receive as input the object-level feature map and configured to output the source determination model output, and one or more feature layers connected to the object feature extraction model to receive as input the object-level feature map and configured to output the one or more object feature vectors. This may improve overall waste unit classification by providing for a more compact waste classification pipeline with fewer processing steps. In some embodiments, the object feature extraction model may be a backbone network (e.g. a feature extracting portion of a trained neural network). In some embodiments the object feature extraction model may be a trained machine learning model and / or a trained neural network. In some embodiments, the object feature extraction model may be one or more layers of a trained machine learning model. In some embodiments, the one or more source classification layers and / or the one or more feature layers may be one or more layers of a machine learning model. In some embodiments, the one or more source classification layers and / or the one or more feature layers may be one or more layers of a neural network.

[0013] In some embodiments, the source is a type, class, specification, source of manufacture, method of manufacture, entity, and / or brand. By determining a source that is a type, class, specification, source of manufacture, method of manufacture, entity, and / or brand, waste may be identified as containing waste units originating from each different source. For example, the source of an object in the waste may be determined as originating from a specific brand and / or specification, and consumption rates of that particular waste product may be determined.

[0014] The method may further comprise updating, based upon the one or more object feature vectors and their corresponding waste unit classifications, the waste identifier database. By updating the waste identifier based upon the one or more feature vectors and their corresponding waste unit classifications, the waste identifier database may be automatically improved upon as soon as waste unit classification is achieved. In this way, the waste identifier database may be iteratively improved upon over time without the need for human interaction and / or review. In some embodiments, the updating may comprise storing the one or more object feature vectors and their corresponding waste unit classifications in the waste identifier database. In some embodiments, the updating may comprise transmitting the one or more object feature vectors and their corresponding waste unit classifications to the waste identifier database.

[0015] The method may further comprise, for one of the one or more objects depicted in the one or more images, determining, based upon the corresponding object source score, that an object source cannot be determined. For example, it may not be possible to determine an object source. Thus, by determining that an object source cannot be determined, further processing steps may be initiated in response. In some examples, determining that an object source cannot be determined may comprise outputting an object source score. In some example, determining that an object source score cannot be determined may comprise comparing the object source score with a criterion.

[0016] The method may further comprise updating, based upon candidate data comprising one or more candidate object source scores and / or one or more candidate object feature vectors, one or more waste unit classes stored in a waste identifier database. That is, as new images are obtained by the waste monitoring unit, these images may be processed in order to update the waste identifier database. For example, the new images may be assigned to a waste unit class in the waste identifier database. In this way, new images obtained by the waste monitoring unit may contribute to the processing and identification of subsequent waste.

[0017] In some embodiments, updating the waste identifier database comprises displaying an image of a candidate object on a display of a user interface, the candidate object corresponding to one of the one or more candidate object source scores and / or one of the one or more candidate object feature vectors, receiving, from a user by the user interface, an indication of one or more waste unit classifications for the candidate object, each waste unit classification corresponding to one of the one or more waste unit classes in the waste identifier database, and assigning, based on the indication, the candidate object to the one or more waste unit classes indicated by the classification indication.

[0018] Providing depictions of candidate objects to a user for waste unit classification by the user may provide a more robust and representative waste identifier database. In more detail, objects originating from unknown provenance may be classified in the case that a waste unit classification cannot be achieved using the waste identifier database alone. The waste identifier database may therefore be updated according to a classification provided by the user, which improves the waste identifier database during subsequent waste unit classification. In some embodiments, the display may be a computer monitor, laptop display, phone screen, tablet display, or any other suitable display. In some embodiments, the user interface may comprise one or more interactive elements that enable the user to provide the indication of the waste unit classification to which the candidate object belongs to the user interface. In some embodiments, the user provides their indication by clicking, touching, speaking, and / or any other method of interacting with the user interface. In some embodiments, the assigning of the candidate object to one or more waste unit classifications may comprise generating candidate data. In some embodiments, the candidate data comprises the one or more candidate object feature vectors and corresponding data indicating the waste unit classification for the respective candidate object.

[0019] In some embodiments, querying the waste identifier database further comprises determining a similarity value between each object feature vector and one or more waste unit cluster centroids, each waste unit cluster centroid corresponding to a different one of the one or more waste unit classes, determining the waste unit classifications based upon whether the one or more similarity values meet a similarity criterion. By determining a similarity value for each one of the one or more object feature vectors when querying the waste identifier database, and further determining the classification of the waste using a threshold value, efficient waste unit classification is provided for. The similarity criterion may be, for example, a threshold. For example, using a threshold value for classification of the waste is straightforward to implement and requires a relatively low number of computational resources. Furthermore, determining a similarity value using the one or more object feature vectors may be more computationally efficient than other methods of waste unit classification. In some embodiments, the similarity values may be numerical values. In some embodiments, the similarity values may be a probability value. In some embodiments, the waste unit cluster centroids may be a vector comprising one or more numerical values. In some embodiments, the waste unit cluster centroids may be determined based upon one or more example object feature vectors corresponding to the same waste unit class in the waste identifier database. In some embodiments, the one or more similarity values are cosine similarity values.

[0020] The method may further comprise controlling, based upon the source determination model output and / or the one or more waste unit classifications, one or more mechanical, electrical, and / or electro-mechanical control mechanisms of the material processing facility. Advantageously, by controlling one or more mechanical, electrical, and / or electro-mechanical control mechanisms, the flow of waste in a material processing facility may be controlled based on the specific sources of waste that have been identified. Furthermore, it may be desirable for certain types of waste to be physically processed differently than other types of waste in the material processing facility. As such, the source determination model output and / or the waste unit classifications provide data that enable different waste to be physically processed differently in the material processing facility, thereby improving waste management processes. In some embodiments, the control may be the control of one or more electronic devices within the material processing facility. In some embodiments, the control may be control which affects the flow of waste in the material processing facility. In some embodiments, the control may be the control of a motor, conveyor belt, automated actuator, or any other suitable mechanical, electrical, or electro-mechanical control mechanism.

[0021] The method may further comprise outputting, based upon the source determination model output and / or the one or more waste unit classifications, output data indicative of waste production in one or more geographic areas. By outputting data indicative of waste production in or more geographic areas, data relating to the consumption of products in these geographic areas may be inferred. In some embodiments, the geographic areas may be a continent, country, region, state / province, city, landform, or town.

[0022] In some embodiments, the one or more material processing facilities comprises a plurality of material processing facilities each located in a different geographic area, and the method further comprises determining an update to the source determination model and / or the first model, and updating local instances of the source determination model and / or first model at each of the plurality of material processing facilities. In particular, the present techniques enable a distributed method of waste processing. In other words, each of the material processing facilities may process individual streams of waste, and query and contribute to a single waste unit classification database. Similarly, each of the material processing facilities may use the same waste classification pipeline. That is, each material processing facility may use the same machine learning models, each machine learning model updated by data obtained from each of the material processing facilities. By having one or more material processing facilities located in different geographic areas, waste received and processed according to the claimed method provides for more representative data on the production of waste in different geographic areas. In some embodiments, the geographic areas may be a continent, country, region, state / province, city, landform, or town.

[0023] The method may further comprise determining that a plurality of images obtained by the waste monitoring unit each include a respective depiction of a same object, providing data representing each of the respective depictions to the first model to output data indicating the same object, and where outputting the one or more object source scores for the at least one object further comprises providing the data indicating the same object to the source determination model, outputting a respective source determination model output for each depiction of the same object, and determining a single object source score for the same object based on the plurality of respective source determination model outputs.

[0024] That is, the method may include determining if any of the depictions of an object is a duplicate. The method may then include a deduplication stage whereby the object source scores are determined for each unique object in the waste. Advantageously, the efficiency of waste unit classification is thereby improved upon by reducing the number of processing steps required for duplicate depictions of the objects in the waste. In some embodiments, the plurality of images may be two or more images captured of the waste depicting two or more of the same object in the waste. In some embodiments, determining that a plurality of images obtained by the waste monitoring unit each include a respective depiction of a same object may comprise generating an object identifier for each object depicted in the one or more images using the data indicating at least one object. In some embodiments, the generating of the one or more object identifiers may comprise providing the one or more object classifications and the data indicating at least one object to a tracking algorithm. In some embodiments, the tracking algorithm may be a Hungarian tracking algorithm. In some embodiments, the tracking algorithm may output an object identifier for each object depicted in the one or more images in response to receiving the one or more object classifications and / or the data indicating the at least one object. In some embodiments, determining the object source score for the same object based upon the plurality of source determination model outputs may comprise comparing their respective object identifiers. In some embodiments, the one or more object classifications may be one or more object material classifications.

[0025] In some embodiments, determining the object source score for the same object comprises averaging the plurality of respective source determination model outputs. Averaging each of the plurality of source determination model outputs provides for a more robust and generalised source determination model output. In some embodiments, the averaging comprises a simple arithmetic mean, a weighted average, a geometric mean, or a harmonic mean.

[0026] The method may further comprise determining that a plurality of images obtained by the waste monitoring unit each include a respective depiction of a same object, providing data representing each of the respective depictions to the first model to output data indicating the same object, where determining the one or more object feature vectors indicative of the at least one object further comprises determining, based upon the data indicating the same object, a respective object feature vector for each depiction of the same object, and determining an object feature vector for the same object based on the plurality of respective object feature vectors.

[0027] That is, the method may include determining if any of the depictions of an object is a duplicate. The method may then include a deduplication stage whereby the object feature vectors are determined for each unique object in the waste. Advantageously, the efficiency of waste unit classification is thereby improved upon by reducing the number of processing steps required for duplicate depictions of the objects in the waste. In some embodiments, the plurality of images may be two or more images captured of the waste containing two or more of the same object depicted in the waste. In some embodiments, determining that one of the at least one object is depicted in the plurality of images may comprise generating an object identifier for each object depicted in the one or more images using the data indicating at least one object. In some embodiments, the generating of the one or more object identifiers may comprise providing the one or more object classifications and the data indicating at least one object to a tracking algorithm. In some embodiments, the tracking algorithm may be a Hungarian tracking algorithm. In some embodiments, the tracking algorithm may output an object identifier for each object depicted in the one or more images using the at least one object depicted in the waste in response to receiving the one or more object classifications and / or the data indicating the at least one object. In some embodiments, determining the object feature vector for the same object based upon the plurality of object feature vectors may comprise comparing their respective object identifiers.

[0028] In some embodiments, determining the object feature vector for the same object comprises averaging the plurality of respective object feature vectors. Averaging each of the plurality of object feature vectors provides for a more robust and generalised representation of one or more objects in depicted in the waste. In some embodiments, the averaging comprises a simple arithmetic mean, a weighted average, a geometric mean, or a harmonic mean.

[0029] There is also described herein a system comprising one or more servers, the one or more servers comprising a waste identifier database and one or more processors. The system further comprises one or more material processing facilities each configured to receive waste and each comprising one or more monitoring units, the monitoring units each comprising one or more cameras and one or more processors, each monitoring unit being configured to carry out the method as set out above.

[0030] There is also described herein a computer readable storage medium comprising computer readable instructions, the computer readable instructions, when executed by a processor, cause the processor to carry out the method as set out above. Brief Description of the Drawings

[0031] Detailed discussion directed to one of ordinary skill in the art is set forth in the specification, which makes reference to the appended figures, in which:

[0032] Figure 1 depicts an example system of geographically distributed material processing facilities each comprising a plurality of monitoring units.

[0033] Figure 2 depicts an example flow of waste through a material processing facility and monitoring unit.

[0034] Figure 3 depicts an example material recognition model for determining a waste material.

[0035] Figure 4 depicts an example source determination model for determining a waste source.

[0036] Figure 5 depicts a flow diagram of a first example waste classification pipeline.

[0037] Figure 6 depicts a flow diagram of a second example waste classification pipeline.

[0038] Figure 7 depicts a flow diagram of a waste unit classification method.

[0039] Figure 8 is a schematic illustration of an exemplary computer system on which aspects described herein may be implemented.

[0040] Reference numerals that are repeated across plural figures are intended to identify the same features in various implementations. Detailed Description

[0041] Unique codes or identifiers may be used to identify and / or track individual products. In the context of the present invention, the term “products” refers to waste products. In waste management, these unique codes or identifiers are useful for decision making and provide useful, meaningful, and valuable information about the waste under management. In particular, these unique codes or identifiers may enable companies to track the flow of products throughout a supply chain, identify areas where waste reduction efforts can be focused, and improve sustainability practices. By identifying and / or tracking individual products with reference to their corresponding unique code or identifier, companies can make informed decisions about stocking and ordering products, thereby reducing the amount of waste generated from overstocking or expired products. Additionally, unique codes or identifiers can be used to track the origin of waste materials, enabling companies to better understand the sources of waste and develop more effective strategies for waste reduction and environmental sustainability. In the context of a material processing facility, unique codes or identifiers and the information they provide may enable the proper processing and / or control of the flow of waste though the facility. In the present context, these unique codes or identifiers may be referred to as stock keeping units (SKUs).

[0042] SKUs may comprise a unique identifier corresponding to a particular product and / or class of product (e.g. a waste product and / or a class of waste product). SKUs may comprise data, in particular numerical values indicating that a product corresponds to a particular product and / or class of product. SKUs may identify the source of the particular waste product. For example, the source of the particular product may be type, class, product specification, source of manufacture, method of manufacture, entity (e.g. manufacturer, distributor, importer), brand, and / or any other data indicative of the source / origin of the product. Each SKU is unique insofar as there is only one SKU for each different product and / or class of product. For example, a SKU may correspond to a particular brand or model of drinks container. In another example, the SKU may correspond to a particular specification of plastic microwavable container. In another example, the SKU may correspond to a particular material. Waste may contain one or more objects or products each corresponding to a SKU. The waste may comprise one or more products each corresponding to the same or different SKU. As will become readily apparent, the methods and systems described herein enable automatic and efficient classification of waste into one or more waste unit classifications, which may correspond to one or more stock keeping units.

[0043] Figure 1 schematically depicts an example system in which techniques described herein may be implemented. In particular, the example system comprises a first material processing facility 106a, a second material processing facility 106b, and a third material processing facility 106c (collectively material processing facilities 106). The first material processing facility 106a comprises three monitoring units 108a, 108b, 108c, the second material processing facility 106b comprises three monitoring units 108d 108e, 108f, and the third material processing facility 106c comprises three monitoring units 108g, 108h, 108i (collectively monitoring units 108). In the example system of Figure 1, the first material processing facility 106a receives first waste 104a, the second material processing facility 106b receives second waste 104b, and the third material processing facility 106c receives third waste 104c. The example system further comprises a waste identifier database 100. While shown as a single database, it will be appreciated that the waste identifier database 100 may be implemented by any number of databases. The waste identifier database 100 may comprise example data and corresponding classification data, the example data being data indicative of an object and the classification data being data indicative that the object belongs to one or more waste unit classes. For example, each waste unit class may correspond to a particular SKU. The classification data may be data indicating that the example data corresponds to a particular cluster. The example data may further comprise one or more timestamps indicative of a point in time in which the example data was generated. Further detail regarding the example data and classification data will become readily apparent with reference to Figure 5 and any description thereof. The waste identifier database 100 may further comprise data representing one or more machine learning models. For example, the one or more machine learning models may be configured for use within a portion of a waste classification pipeline. The waste classification pipeline is described with reference to Figures 2 and 5 below. The waste identifier database 100 may be communicatively coupled with each one of the monitoring units such that data may be passed to / from the waste identifier database 100 and each of the monitoring units 108. Each of the monitoring units 108 may comprise computing devices with a network interface that enables the upload and download of data to and from the waste identifier database 100. The upload may include the upload of data indicative of waste being processed through a given monitoring unit 108. Download of data from the waste identifier database 100 may include the download of one or more machine learning models for use by any one of the monitoring units 108. The upload and download of data to / from the waste identifier database 100 may be initiated by any one of the monitoring units 108, or the waste identifier database 100 itself. For example, a monitoring unit 108 may upload data to the waste identifier database 100 at request of the monitoring unit 108 itself or at request of the waste identifier database 100. In another example a monitoring unit 108 may download data from the waste identifier database 100 at request of the monitoring unit 108 itself or at request of 100.

[0044] Advantageously, the waste identifier database 100 may make available one or more machine learning models to any one of the monitoring units 108108 in an efficient, centralised, and standardised manner. In such a way, processing on each of the monitoring units occurs in accordance with the one or more machine learning models of the waste identifier database 100, thereby providing predictable classification of objects in the waste with regards to each one of the monitoring units 108. Furthermore, any improvements made to the one or more machine learning models stored in the waste identifier database 100 may be propagated (e.g. downloaded on instruction) to each one of the monitoring units 108. In this way, each of the monitoring units 108 may quickly and consistently benefit from improvements made to the one or more machine learning models stored in the waste identifier database 100.

[0045] The example system of Figure 1 may further comprise one or more waste management tools 102. The one or more waste management tools 102 may be provided by any appropriate computing device, such as a personal computer, a computer, a phone, a tablet, and / or one or more servers. The one or more waste management tools 102 may be connected remotely over a network. The one or more waste management tools 102 may comprise a user interface configured to provide a user with access to any one of the one or more waste management tools 102. The one or more waste management tools 102 may comprise a query tool 110 for querying the waste identifier database 100 for data indicative of one or more waste unit classifications based on query data indicative of the waste 104, a tracking tool 112 configured for tracking the flow of waste through any one of the material processing facilities 106 over time, a visualization tool 114 configured for visualizing data relating to the flow of waste through any one of the material processing facilities 106a 106b 106c, and a database update tool 116 for updating the waste identifier database 100. The updating of the waste identifier database 100 may comprise updating the example data, classification data, and / or data representing one or more machine learning models in the waste identifier database 100.

[0046] Querying the waste identifier database 100 for data indicative of one or more waste unit classifications based on query data indicative of the waste may comprise comparing the query data to the example data and / or classification data in the waste identifier database 100. For example, the query data may be indicative of a query for searching the waste identifier database 100 for example data and corresponding classification data belonging to a particular waste unit class. Tracking the flow of waste over time may comprise outputting data indicative of one or more waste unit classifications at one or more different points in time based upon the one or more corresponding timestamps of the example data. Visualizing data relating to the flow of waste comprises outputting visual representations of the example data and / or the classification data in the waste identifier database 100 on a display. Updating may further comprise uploading new example data and corresponding new classification data to the waste identifier database 100. The uploading may be initiated by any one of the monitoring units 108, or manually by a user. Updating may further comprise a user interacting with a user interface (e.g. a graphical user interface) implementing the database update tool 116. The user interface may be configured to display example data to a user and enable a user to assign classification data to the example data in the waste identifier database 100. For example, the user interface implementing the database update tool 116 may allow a user to view an image representing an object in the waste 104 corresponding to new example data, assign new classification data to the new example data based upon a decision of the user viewing the image, and upload the new example data and the corresponding new classification data to the waste identifier database 100. While three material processing facilities 106 are depicted in Figure 1, this is merely exemplary and that there may be any number of material processing facilities. The material processing facilities 106 may be distributed across a geographical area. The geographical area may be a geographical area of any size and may be large. The geographical area may be, by way of example only, a town, a city, a state, or a country. Likewise, each material processing facility 106 may each comprise any number of monitoring units 108 and may each comprise a different number of monitoring units 108 to other material processing facilities 106, where three monitoring units are shown in Figure 1 by way of example only. For example, ten material processing facilities may be provided across the United Kingdom with two located in Manchester, one located in Liverpool, two located in Reading, three located in Bristol, and two located in Edinburgh. In this example, those located in Manchester, Liverpool, and Reading may comprise four monitoring units each and those located in Bristol and Edinburgh may comprise two monitoring units each. It will be understood that any suitable arrangement is possible.

[0047] The first waste 104a, the second waste 104b, and the third waste 104c (collectively waste 104) may each represent a waste stream originating from one or more locations at one or more points in time. For example, the first waste 104a may comprise waste originating from Manchester on a first date and waste originating from Liverpool on aa second date. The first waste 104a may be received at the material processing facility 106a. The second waste 104b may be received at the material processing facility 106b. The third waste 104c may be received at the material processing facility 106c. The waste 106 may be received at the respective material processing facilities by any appropriate means (e.g. by truck).

[0048] Upon receiving the waste 104, the material processing facility 106 may begin to process the waste 104. The material processing facility 106 may process the waste 104 using one or more monitoring units 108. For example, the first material processing facility 106a may process the first waste 104a using the first monitoring unit 108a. In another example, the first waste 104a may be processed at the first material processing facility 106a by three separate monitoring units 108a 108b 108c. For example, the first waste 104a may be separated into three separate batches, each batch for processing by a different one of the three monitoring units 108a 108b 108c.

[0049] As will become apparent, the monitoring units 108 may each be configured to receive as input an image and generate as output waste identification data. Additionally, the monitoring units 108 may each communicate with the waste identifier database 100. The communication may be over a network and may be facilitated by a network interface of the monitoring unit communicating with a network interface of the waste identifier database 100. The network may be a virtual private network (VPN), internet, or any other suitable network. The monitoring units 108 may each communicate data indicative of a respective waste 104 being processed. For example, data indicative of the first waste 104a processed through the first material processing facility 106a may be communicated to the waste identifier database 100. The data indicative of waste processed through a material processing facility may be identification data and may, for example, be waste unit classification data corresponding to one or more SKUs. For example, the data indicative of waste processed through a material processing facility may be data indicative that a particular number of objects having a particular identifier (for example, a particular SKU) were processed through the material processing facility. The data indicative of waste processed through a material processing facility may also be indicative that the waste was processed through a particular monitoring unit of a particular material processing facility.

[0050] In such a way, the wide distribution and variation of waste products in a given geographic area (i.e. the area covered by each of the material processing facilities) may be fully appreciated. This distribution can depend on a number of factors such as seasonality, regulatory variations, localization and / or internationalization of consumer packaging, and / or production / supply chain / consumption rates.

[0051] The waste identifier database 100 may be a database stored in a central location or distributed across a number of locations. The waste identifier database 100 may be a cloud-hosted database. The waste identifier database 100 may comprise a network interface enabling the waste identifier database 100 to make requests, be queried, be searched, and / or be updated as will be understood by those skilled in the art. The waste identifier database 100 may further comprise one or more machinelearning models. The waste identifier database 100 may be configured to propagate the one or more machine-learning models to the monitoring units 108.

[0052] Figure 2 schematically depicts an example system operating at a material processing facility in which techniques described herein may be implemented. In particular, techniques described herein may be used for the processing and identification of waste at a material processing facility. The example system comprises a monitoring unit 200, a waste classification pipeline 202, one or more servers 206, a conveyor belt 212, and waste 210 comprising a plurality of waste units. A waste unit in the waste 210 may be an object, product, or any other unit in the waste 210. The monitoring unit 200 comprises a camera 204 and a first network interface 214. The one or more servers 206 comprise a waste identifier database 208 and a second network interface 216. The first network interface 214 enables the monitoring unit 200 to communicate with other computing devices (e.g. the one or more servers 206) over a network. The second network interface 216 enables the one or more servers 206 to communicate with other computing devices (e.g. the monitoring unit 200) over a network. As explained in more detail below, the waste 210 may be processed by the monitoring unit 200 in such a way that enables efficient and accurate waste unit classification. Furthermore, the implementation of a centralised waste identifier database 208, the querying and updating thereof being enabled by various different aspects of the methods and systems described herein, further contribute to the efficiency and accuracy of waste unit classification across one or more material processing facilities.

[0053] The processing of waste as referred to herein may be exemplified with reference to Figure 2. In the example of Figure 2, the waste 210 is received at a material processing facility. The waste 210 may then be processed by the monitoring unit 200. The processing may comprise the camera 204 outputting one or more images of the waste 210. The one or more images of the waste 210 may be output by the camera 204 as the waste transitions into view of the camera 204 along the conveyor belt 212. It will be appreciated that the camera 204 may be any device capable of imaging the waste 210 such that the resulting images of the waste 210 are capable of being input into the waste classification pipeline 202 as described below. Conventionally, the camera 204 may comprise a visible-light camera capable of generating images in both full colour and / or greyscale. However, the camera 204 may additionally or alternatively comprise other imaging technologies such as an infrared camera configured to generate an infrared image, LIDAR sensors, etc. As such, the one or more images of the waste may be the output of any suitable sensor. The camera 204 may be configured to operate at a specific frame rate. The specific refresh rate may be specific to the operating speed of the conveyor belt 212, or any other relevant condition of the methods or systems described herein. The refresh rate may determine how many images of the waste 210 are output. It will be appreciated that any other suitable means for transitioning the waste 210 into the view of the camera 204 may be possible.

[0054] As will become apparent with reference to the description below, the waste classification pipeline 202 may comprise one or more processes, algorithms, and / or machine learning models configured to, as a collective, receive as input an input image and output one or more classifications of the waste 210 based on one or more waste identifiers. These classifications may indicate that the waste comprises one or more objects (e.g. products). The classifications may each correspond to one or more stock keeping units indicating that the object belongs to those one or more stock keeping units. The waste classification pipeline 202 may comprise multiple portions and may be partitioned. For example, one portion of the waste classification pipeline 202 may be implemented on the monitoring unit 200, while another portion of the waste classification pipeline 202 may be implemented on the one or more servers 206. Intermediate data in the waste classification pipeline 202 may be transmitted between the monitoring unit 200 and the one or more servers 206 over a network, facilitated by the first network interface 214 and the second network interface 216.

[0055] For example, the monitoring unit 200 may input image data into the waste classification pipeline 202, and the waste classification pipeline 202 may generate first intermediate data as output from a first portion of the waste classification pipeline 202 implemented on the monitoring unit 200. In this example, the first intermediate data may subsequently be transmitted to the one or more servers 206 by the first network interface 214 of the monitoring unit 200 over a network. In this example, a second portion of the waste classification pipeline 202 is implemented on the one or more servers 206. In this example, the first intermediate data is then received as input into the second portion of the waste classification pipeline 202 and processing in the waste classification pipeline 202 continues.

[0056] In another example, first intermediate data generated by the monitoring 200 and transmitted to the one or more server 206 may be used to output second intermediate data from a second portion of the waste classification pipeline 202 implemented on the one or more servers 206. In this example, the second intermediate data may subsequently be transmitted to the monitoring unit 200 by the second network interface 216 of the one or more servers 206 over a network. In this example, a third portion of the waste classification pipeline 202 is implemented on the monitoring unit 200. In this example, the second intermediate data is then received as input into the third portion of the waste classification pipeline 202 and processing in the waste classification pipeline 202 continues. Implementations of the waste classification pipeline 202 are described in further detail with reference to Figure 5 below.

[0057] Figure 3 schematically depicts a material recognition model 300. The material recognition model 300 may comprise a feature extraction model 304, an object detector 308, and an object classifier 310. The material recognition model 300 enables the localization and classification of objects (e.g. material) in an image. For example, the material recognition model 300 may localize an object in the top-right of an image and classify the object as “metal”. Advantageously, the material recognition model 300 may be used to automatically and efficiently detect the specific location and classification of objects in one or more images. In such a way, systems and methods requiring such localization and classification of objects may be improved by incorporating the material recognition model 300 described herein. As referred to herein, material refers to any physical matter. The material may have specific physical characteristics and / or appearances. For example, the term material may refer to metal, wood, plastic, combination or any sub-category thereof (e.g. aluminium, oak, and polyethylene). The material recognition model 300 may be able to localize and recognize objects within an image based upon the physical characteristics and / or appearances of the object (e.g. a material the object is comprised of).

[0058] The material recognition model 300 may be a trained machine learning model. The material recognition model 300 may be configured to receive as input an input image 302. The input image 302 may be an image file or other data representing an image. For example, the input image 302 may be an image file or a vector representation of an image. The material recognition model 300 may be configured to output object coordinates 312 and object classifications 314. The object coordinates 312 may be data which indicates the location of an object in an image. The object coordinates 312 may be one or more numerical values indicating a point or region in which an object is located in an image. The object coordinates 312 may be a vector representing the one or more numerical values indicating a point or region in which the object is located in an image. For example, the object coordinates 312 may be a vector indicating a number of points in the image. For example, each point may be represented by two (2) numerical values within the vector. In one example, the object coordinates 312 may comprise eight (8) numerical values, thereby defining four (4) points in the image. In this example, the numerical values indicate a region in the image (e.g. a bounding-box) in which the object is located. The object coordinates 312 may be relative to a coordinate system of an image. The object classifications 314 may be one or more numerical values indicating a classification of an object in an image. The classification of an object in an image may be a classification of the object as comprising one or more materials. The object coordinates 312 and object classifications 314 may correspond with each other. For example, a first object coordinate of the object coordinates 312 may correspond with a first object classification of the object classifications 314, and a second object coordinate of the object coordinates 312 may correspond with a second object classification of the object classifications 314. The object coordinates 312 may comprise one or more object coordinates, and that the object classifications 314 may comprise one or more corresponding object classifications.

[0059] The material recognition model 300 may comprise one or more trained machine learning models. These trained machine learning models may be trained neural networks. The feature extraction model 304, the object detector 308, and the object classifier 310 may each be a trained machine learning model. For example, the feature extraction model 304, the object detector 308, and the object classifier 310 may each be a fully connected neural network, a convolutional neural network, a vision transformer, a Siamese neural network, or any other machine learning model consistent with the description herein. In another example, the feature extraction model 304, the object detector 308, and the object classifier 310 may each be a ResNet model, a VGG model, an EfficientNet model, or any other machine learning model consistent with the description herein. In one example, the feature extraction model 304 may be a backbone network (e.g. a pre-trained neural network configured to extract features from a given input). Each one of the feature extraction model 304, the object detector 308, and the object classifier 310 may be trained using one or more training examples. Alternatively, the feature extraction model 304, the object detector 308, and the object classifier 310 may all be different parts of a single machine learning model, the single machine learning model being the material recognition model 300. It will be readily appreciated that different configurations of the material recognition model 300 may be envisaged.

[0060] In one example, the feature extraction model 304 may be trained using one or more training examples, each one of the training examples comprising an input image and corresponding ground truth data. Any appropriate training method may be used. For example, backpropagation may be used to update the parameters of the feature extraction model 304 to minimize a loss associated with either a given training example or a batch of given training examples. The loss may be a difference between an output of the feature extraction model 304 and the corresponding ground truth data for the given training example. Any appropriate loss may be used as will be apparent to the skilled person. By way of example only, the loss may be a mean squared error (MSE), a binary cross-entropy loss, a categorical cross-entropy loss, or any other suitable loss function. In another example, the feature extraction model 304 may be a part of a larger trained machine learning model. The larger trained machine learning model may have been trained to classify input images and / or perform any other machine learning task associated with the image. The object detector 308 and / or the object classifier 310 may also be trained in a corresponding manner using training examples associated with corresponding ground truth data. Alternatively, the object detector 308 and / or the object classifier 310 may be a part of a larger trained machine learning model. In the alternative, the material recognition model 300 may be trained as a single machine learning model comprising one set of parameters.

[0061] Upon receiving the input image 302 at the material recognition model 300 as input, the input image 302 may be processed through the feature extraction model 304 to output a feature map 306. The feature map 306 may comprise one or more numerical values representing the input image 302. The feature map 306 may be a vector representation of the input image 302. The feature map 306 may represent the visual features of the input image 302 in one or more numerical values. The feature map 306 may be an image-level feature map. The feature map 306 may be a latent feature vector in the material recognition model 300. For example, the feature map 306 may serve as a latent representation of the input image 302 in the material recognition model 300.

[0062] The feature map 306 may be provided as input to both the object detector 308 and the object classifier 310. The object detector 308 may be configured to receive as input the feature map 306 and output the object coordinates 312. The material recognition model 300 may output the object coordinates 312 upon output of the object coordinates 312 from the object detector 308. The object classifier 310 may be configured to receive as input the feature map 306 and output the object classifications 314. The material recognition model 300 may output the object classifications 314 upon output of the object classifications 314 from the object classifier 310. For example, the object detector 308 and the object classifier 310 may each be implemented by one or more final layers of the material recognition model 300.

[0063] The material recognition model 300 may be implemented on one or more computing devices. All or part of the material recognition model 300 may be implemented on any one of the monitoring units 108 (i.e. on the edge) as described herein with reference to Figure 1 and Figure 2. In such a way, processing by the material recognition model 300 on the edge means that processed occurs in real-time. Furthermore, bandwidth and / or latency limitations which may be associated with transmitting data to the cloud are not experienced with on the edge processing. Alternatively, the material recognition model 300 may be implemented on one or more servers separate from the monitoring units 108 (i.e. on the cloud). In such a way, processing by the material recognition model 300 on the cloud means that data transmitted to the cloud for processing may be stored for future historical analysis and / or other use. Furthermore, processing on the cloud enables the implementation of more computationally efficient and / or powerful computer hardware and / or methods used for processing which may not otherwise be available on the edge.

[0064] Figure 4 schematically depicts a diagram of a source determination model 400. The source determination model 400 may comprise an object feature extraction model 408, one or more source classification layers 410, and one or more feature layers 412. The source determination model 400. The determination is configured to output of a prediction of a source of an object in an input image 402. The source determination model 400 may further output a feature vector representing the object depicted in an input image 402. The input image 402 may be the input image 302 of Figure 3. Advantageously, the source determination model 400 may be used to automatically and efficiently output a feature vector representation of one or more objects in an image. Where the source determination model 400 outputs a vector representation of the object, this may be used for further processing, as described in more detail below. As referred to herein, source refers to a person, or thing from which an object emanates or originates. The source may be a real or legal person (e.g. a company) or their brand, a device (e.g. a manufacturing device), or any other person or thing. In some cases, the source may be indicative of a geographical location.

[0065] The source determination model 400 may be a trained machine learning model. The source determination model 400 may be configured to receive as input a portion of an input image 406. It will be readily appreciated that the source determination model 400 may generally be configured to receive the input image 402 as input. However, for the purpose of exemplifying the techniques described herein, the source determination model 400 is described as being configured to receive a portion of the input image 402 specifically. The portion of an input image 406 may be a cropped input image. The portion of an input image 406 may be an image representing an object in the input image 402. The portion of an input image 406 may be a subset of pixels of the input image 402. The portion of an input image 406 may be one of a plurality of portions of an input image associated with the input image 402. The portion of an input image 406 may be generated based on the input image 402 and one or more object coordinates 404. The one or more object coordinates 404 may indicate one or more portions of the input image 402 in which an object is located, the portion of an input image 406 being one portion of the input image 402 representing one object located in the input image 402. The one or more object coordinates 404 may be the object coordinates 312 of Figure 3. The portion of an input image 406 may be generated prior to being input into the source determination model 400 by any suitable computing device implementing the systems and methods described herein. The source determination model 400 may receive a plurality of portions of an input image as input, and output a plurality of respective source determination model outputs.

[0066] For each portion of the input image 406 provided as input, the source determination model 400 may be configured to output an object source score 414 and an object feature vector 416. The object source score 414 may be data which indicates a likelihood that a source of an object depicted in the portion of an input image 406 may be determined. More specifically, the object source score 414 may be a single probability that a source of an object depicted in the portion of an input image 406 originates from a particular source, or whether it originates from a generic source. The object source score 414 may be a numerical value. For example, the object source score 414 may be a numerical value within a specific range (e.g. 0 to 1 or -1 to 1). The numerical value maybe probability value indicating the likelihood that a source of an object depicted in the portion of an input image 406 may be determined. In one example, the object source score 414 may be 0.1 indicating a low likelihood that a source of an object depicted in the portion of an input image 406 may be determined, and as a consequence, that there is a high likelihood that the depicted object originates from a generic source (e.g. the object is not branded).

[0067] In other embodiments, the object source score 414 may be a vector comprising a single numerical value indicating a likelihood that a source of an object depicted in the portion of the input image 406 may be determined. In other embodiments, the object source score 414 may be one numerical value in a “one-hot” (or “zero-hot”) vector, the “one-hot” vector comprising at most one positive value for a single possible source, and zero values for all other possible sources. In other embodiments, the object source score 414 may be one numerical value in a “multi-hot” vector, the “multi-hot” vector comprising a positive (or zero) value for each possible source. The object feature vector 416 may be a vector indicative of the object depicted in the portion of the input image 406. In one example, the object feature vector 416 is a vector comprising 128 numerical values and having a shape (1, 128). In another example, the object feature vector 416 is a vector comprising 192 numerical values and having a shape (3, 64). The object feature vector 416 may take any appropriate form.

[0068] The object feature extraction model 408 may be a trained machine learning model or may be one or more layers of a trained machine learning model. For example, the object feature extraction model 408 may be a fully connected neural network, a convolutional neural network, a vision transformer, a Siamese neural network, or any other machine learning model consistent with the description herein. In another example, the object feature extraction model 408 may be a ResNet model, a VGG model, an EfficientNet model, or any other machine learning model consistent with the description herein. In one example, the object feature extraction model 408 may be a backbone network (e.g. a pre-trained neural network configured to extract features from a given input). The object feature extraction model 408 may be configured to receive as input the portion of an input image 406 and to output a vector representation of the portion of an input image 406. The vector representation of the portion of an input image 406 may be an object-level feature map. The one or more source classification layers 410 may comprise one or more layers of a trained machine learning model (e.g. a trained neural network) configured to output the object source score 414. The one or more feature layers 412 may comprise one or more layers of a trained machine learning model configured to output the object feature vector 416. The one or more source classification layers 410 and the one or more feature layers 412 may be connected to the output of the object feature extraction model 408. The portion of an input image 406 may be received as input by the object feature extraction model 408 and the object feature extraction model 408 may output a vector representation of the portion of an input image 406. The vector representation of the portion of an input image 406 may be a latent vector between the object feature extraction model 408 and the one or more source classification layers 410 and the one or more feature layers 412. The one or more source classification layers 410 may output the object source score 414 based on the vector representation of the portion of an input image 406. The one or more feature layers 412 may output the object feature vector 416 based on the vector representation of the portion of an input image 406. It will be appreciated that while the feature extraction model 304 and the object feature extraction model 408 are depicted and described as separate feature extraction models, this is not an essential feature. In alternative implementations, the object features may be extracted at any stage of the waste classification pipeline. However, by extracting an image-level feature map using a separate feature extraction model 304, redundant calculation of more detailed object-level features may be avoided, thereby making waste classification more efficient and therefore suitable for distributed implementation.

[0069] It will be appreciated that while it is described that the source determination model of Figure 4 takes as input a portion of the input image 406 and processes the portion of the input image 406, in other examples, the source determination model 400 may be configured to receive the input image 402 and to process the input image 402 without receiving or obtaining a portion of an input image. Similarly, the source determination model 400 may be configured to itself determine one or more portions of an input image from an input image 402. For example, the source determination model 400 may be configured to receive object coordinates 404 and to determine one or more portions of the input image 406. Alternatively, the source determination model 400 may be configured to determine object coordinates 404 from a received input image 402. More generally, it will be appreciated that the source determination model 400 may include any or all of the components of the material recognition model of Figure 3. The source determination model 400 and the material recognition model 300 may be implemented by a single machine learning model having a single set of parameters that is trained ‘end-to-end’.

[0070] Figure 5 schematically depicts a diagram of a waste classification pipeline 500. The waste classification pipeline 500 comprises a material recognition model 504, a source determination model 510, a deduplication stage 512, an averaging module 520, and a waste identifier database 526. The material recognition model 504 may be the material recognition model 300 described with reference to Figure 3. In particular, the material recognition model 504 may be configured to receive as input an input image 502, and output one or more object coordinates 506 and one or more corresponding object classifications 508. The source determination model 510 may be the source determination model 400 described with reference to Figure 4. In particular, the source determination model 510 may be configured to receive as input a portion of an input image based on both the input image 502 and the one or more object coordinates 506, and output one or more object source scores 514 and object feature vectors 518. The input image 502 may be the input image 302 described with reference to Figure 3 and / or the input image 402 described with reference to Figure 4. The one or more object coordinates 506 and the one or more object classifications 508 may be the one or more object coordinates 312 and one or more object classifications 314 respectively described with reference to Figure 3. Each one of the one or more object source scores 506 may be the object source score 414 described with reference to Figure 4. Each one of the one or more object feature vectors 518 may be the object feature vector 416 described with reference to Figure 4.

[0071] The deduplication stage 512 may be configured to receive as input the one or more object coordinates 506 and / or the one or more corresponding object classifications, and output one or more corresponding object identifiers 516. The deduplication stage 512 may be configured to output an indication (e.g. by means of the one or more object identifiers 516) that an object in the input image 502 is, or is not, a duplicate. In this way, objects represented in the input image 502 which represent the same real-world object as those which have previously, or are concurrently, being processed through the waste classification pipeline 500 may be identified, thereby avoiding processing the same object more than once. The deduplication stage 512 may comprise a tracking algorithm. The tracking algorithm may be a linear assignment solver. In one example, the tracking algorithm may be a Hungarian algorithm. The deduplication stage 512 need not be present in all implementations of the waste classification pipeline 500. For example, with reference to Figure 2, the deduplication stage 512 may be omitted if the camera 204 which generates the input image 502 operates at frame rate low enough to not produce images that contain duplicate depictions of the same object in a plurality of input images processed by the waste classification pipeline 500, and / or if the conveyor belt 212 operates at a fast enough speed to cause the camera 204 to not capture images containing duplicates in the waste 210, and thereby avoid counting the same object more than once. In another example, the deduplication stage 512 may be omitted from the waste classification pipeline 500 depending on the particular configuration and implementation of the waste classification pipeline 500 (e.g. a particular portion of the waste classification pipeline 500 is implemented on the cloud which may not be compatible with the deduplication stage 512).

[0072] With reference to Figure 2, multiple images of the waste 210 may be captured by the camera 204 for processing in the waste classification pipeline 202 (e.g. the waste classification pipeline 500 of Figure 5). In these images of the waste 210, one or more of the same object may be identified as the waste 210 moves along the conveyor belt 212. In some cases, the same object in the waste 210 may be captured by the camera 204 multiple times thereby resulting in multiple images of the same object being captured and a corresponding multiplicity of processing operations in the waste classification pipeline 202. It is therefore desirable to identify which objects in the waste 210 have been captured in images more than once in order to take this into account during further processing, thereby reducing the computational burden of processing images of the same object more than once. The deduplication stage 512 therefore provides this function by assigning different objects different object identifiers 516.

[0073] The averaging module 520 may be configured to receive as input two or more object source scores 514 and / or two or more object feature vectors 518, and two or more corresponding object identifiers 516 output from the deduplication stage 512. The averaging module 520 may be configured to output one or more unique object source scores 522 and / or one or more unique object feature vectors 524. The one or more unique object source scores 522 may be one or more object source scores that are unique to a given real-world object represented in the input image 502. The one or more unique object feature vectors 524 may be one or more object feature vectors that are unique to a given real-world object represented in the input image 502. In order to output the one or more unique object source scores 522, the averaging module 520 may average the two or more object source scores 514 having the same corresponding object identifier of the one or more corresponding object identifiers 516. In order to determine the one or more unique object feature vectors 524, the averaging module 520 may average the one or more object feature vectors 518 corresponding to objects having the same object identifier 516.

[0074] The averaging may comprise a simple arithmetic mean, a weighted average, a geometric mean, or a harmonic mean. For example, two input images may each depict an instance of an object (e.g. a 330 ml cans of cola) that has a corresponding object identifier “173”. The source determination model 510 may output an object source score of [0.1] and an object feature vector of [1.1, 2.5, -1.3] for a first instance of the object (e.g. in the first input image), and may output an object source score of [0.4] and an object feature vector of [0.9, 2.1,-1.4] for a second instance of the object (e.g. in the second input image). The averaging module 520 may process the object source scores for both instances in order to output a unique object source score of [0.25] and an object feature vector of [1.0, 2.3, -1.35] for the same object having the object identifier “173”. In this example, the object source scores [0.1] and [0.4] are duplicates in the sense that they both correspond to the same object identifier, and by extension, the same real-world object in the input images 502 provided as input to the waste classification pipeline 500. In this example, the object feature vectors [1.1, 2.5, -1.3] and [0.9, 2.1, -1.4] are also duplicates in the sense that they both correspond to the same object identifier, and by extension, the same real-world object.

[0075] Advantageously, the deduplication stage 512 and averaging module 520 may enable redundant data to be reduced. Furthermore, redundant image data for a given object may be used to build a more accurate and / or robust representation of the object during processing. For example, by providing multiple representations of an object positioned at multiple angles, a more generalised and robust representation of the object may be obtained, enabling more accurate processing of the object when seen at different angles. Additionally, because the deduplication stage 512 and averaging module 520 ensure that only one classification event occurs per real-world object in the waste, the rate (i.e. speed) of classification of the waste is increased (e.g. because duplicate objects are not classified). For the same reason, the deduplication stage 512 and averaging module 520 provides for more space efficient methods and systems.

[0076] For the same reasons as the deduplication stage 512, the averaging module 520 need not be present in all implementations. The description of the averaging module 520 referring to receiving two or more object source scores 514 and / or two or more object feature vectors 518 means that averaging occurs when it is determined that a plurality of images provided as input to the waste classification pipeline 500 each include a respective depiction of the same object. In other words, when at least two or more objects with the same corresponding object identifier are identified, averaging may occur for the two or more object source scores 514, the two or more object feature vectors 518, or both.

[0077] The waste identifier database 526 may comprise one or more waste unit classes. The waste unit classes may each correspond to one or more stock keeping units. In one example, the waste unit classes may each correspond to a different stock keeping unit to any other one of the waste unit classes. The one or more waste unit classes may each comprise a waste unit cluster centroid. The waste unit cluster centroid may correspond to a different stock keeping unit. The waste identifier database 526 may comprise example data and corresponding classification data. The example data and its corresponding classification data may be examples of previously processed objects and / or waste products. These examples may be compared with new objects identified in waste and depicted in images processed by the waste classification pipeline 500. The example data may be data indicative of an object and the corresponding classification data may be data indicative that the object belongs to one of the one or more waste unit classes. The example data may comprise one or more object source scores and / or one or more object feature vectors. The corresponding classification data may comprise one or more waste unit classifications. It will be readily apparent that these object source scores, object feature vectors, and waste unit classifications may be respective examples of the object source scores 514, object feature vectors 518, and waste unit classifications 528 previously processed by the waste classification pipeline 500, the waste unit classifications 528 being described in further detail below.

[0078] The classification data may indicate that the example data has been assigned to a particular waste unit cluster centroid in the waste identifier database 526. The example data may have been assigned to the waste unit cluster centroid manually by a user. Alternatively, the waste unit cluster centroid may have been assigned to the example data according to a clustering algorithm. The clustering algorithm may be a K-Nearest Neighbour clustering algorithm, or any other suitable clustering algorithm. The waste identifier database 526 may be updated by uploading the one or more unique object source scores 522 and / or the one or more unique object feature vectors 524 to the waste identifier database 526. Updating the waste identifier database 526 may comprise updating the waste unit cluster centroids. The waste unit cluster centroids may be updated according to a K-Nearest Neighbour clustering algorithm, or any other suitable clustering algorithm. The waste unit cluster centroids may be updated based on the one or more unique object source scores 522 and / or the one or more unique object feature vectors 524.

[0079] Advantageously, updating the waste identifier database 526 provides for a more robust and / or representative waste identifier database 526 that can evolve over time. In such a way, accuracy and efficiency of waste unit classification is improved. In particular, identification of a wide variety of objects may be supported by the continuous and iterative addition of new objects as they’re processed by a waste classification pipeline.

[0080] The waste identifier database 526 may be configured to receive queries and to output one or more waste unit classifications 528. In one example, the one or more waste unit classifications 528 may be one or more classifications of objects depicted in the images processed by the waste classification pipeline 500. In one example, the one or more waste unit classifications 528 each correspond to one or more stock keeping units (SKUs). In particular, the waste identifier database 526 may receive a query comprising one or more object source scores 522 and / or the one or more unique object feature vectors 524. The query may be received by the waste identifier database 526 as part of the waste classification pipeline 500 and may be made by a monitoring unit or by one or more servers. In particular, the query may be made by whichever computing device is implementing the portion of the waste classification pipeline 500 directly preceding the query stage. The query may cause the waste identifier database 526 to output the one or more waste unit classifications 528. The one or more waste unit classifications 528 output by the waste identifier database 526 may indicate that one or more objects in the input image 502 each belong to one or more stock keeping units. With reference to Figure 1, each waste unit classification 528 output by the waste identifier database 526 may indicate that particular product having the waste unit classification 528 has been processed through a particular material processing facility 106 and / or a particular one of its respective monitoring unit(s) 108. Each waste unit classification 528 may correspond to a particular product or object, product specification, and / or brand, and may comprise information relating to product type, size, colour, and / or any other attributes of the product. For example, the input image 502 may depict waste which contains an aluminium drinks container of a particular size and which contained a particular product and originating from company A under a particular brand, and a plastic microwavable container of a particular size that contained a particular product and originating from company B under a particular brand. In this example, the waste identifier database 526 may output waste unit classifications 528 for the particular aluminium drinks can and the plastic container. In this way, it is possible to accurately, and in detail, determine the composition of the waste. That is, in the example above, it is possible to determine information relating material, products, companies, brands, etc. of the items in the waste by a given stock keeping unit.

[0081] The one or more waste unit classifications 528 may be formatted as a vector, a matrix, or a tensor comprising one or more numerical values. The one or more waste unit classifications 528 may each comprise one or more numerical values, each numerical value indicating a probability that an object in the input image 502 corresponds with a particular stock keeping unit. For example, the one or more waste unit classifications 528 may be [[[0.1, 0.2, 0.9], [0.8, 0.15, 0.3]], [[0.1, 0.2, 0.9], [0.8, 0.15, 0.3]]]. In this example, [[0.1, 0.2, 0.9], [0.8, 0.15, 0.3]] (array A) may correspond with a first input image processed through the waste classification pipeline 500, and [[0.1, 0.2, 0.9], [0.8, 0.15, 0.3]] (array B) may correspond with a second input image processed though the waste classification pipeline 500. The first input image and second image may both be processed substantially concurrently. In this example, array A comprises two vectors each corresponding to a different object or product in the input image 502. These two vectors each comprise three probability values indicating a probability that the respective object or product belongs to each of three corresponding stock keeping units.

[0082] In some embodiments, the one or more waste unit classifications 528 may comprise one or more similarity values, each similarity value indicating the similarity between one of the one or more object feature vectors 518 and a waste unit cluster centroid in the waste identifier database 526. In one example, the similarity value may be a cosine similarity value. For example, the cosine similarity value may be a value in the range between 0 and 1, the value of 1 indicating a complete similarity and the value of 0 indicating a complete dissimilarity. In some embodiments, the one or more waste unit classifications 528 may be compared to a threshold value in order to determine a discrete value for the one or more waste unit classifications 528. For example, the threshold value may be 0.8. In the example that the one or more waste unit classifications 528 are [0.21, 0.89], the threshold value of 0.8 may be used to determine that the one or more waste unit classifications 528 are [0, 1], In this example, the value of 0 indicates that a first object in the input image 502 is not classified as a waste unit class corresponding to the first index of the one or more waste unit classifications 528, and that the first object in the input image 502 is classified as a waste unit class corresponding to the second index of the one or more waste unit classifications 528.

[0083] Alternatively, the one or more waste unit classifications may comprise one or more numerical values, each numerical value indicating a waste unit cluster centroid to which an object in the input image 502 has been assigned, the waste unit cluster centroid corresponding to a particular stock keeping unit. For example, the one or more waste unit classifications 528 may be [[10, 3, 95], [24, 10, 96]] In this example, [10, 3, 95] (vector A) may correspond with a first input image processed through the waste classification pipeline 500, and [24, 10, 96] (vector B) may correspond with a second input image processed though the waste classification pipeline 500. Again, the first input image and second image may be processed substantially concurrently. In this example, vector A comprises three numerical values indicating that a first object in the first input image has been assigned to a waste unit cluster centroid “10” corresponding to a first stock keeping unit, that a second object in the first input image has been assigned to a waste unit cluster centroid “3” corresponding to a second stock keeping unit, and that a third object in first input image has been assigned to a waste unit cluster centroid “95” corresponding to a third stock keeping unit. In this example, in vector B, the three numerical values indicate that first, second, and third objects in the second input image have been assigned to the waste unit cluster centroids “24”, “10”, and “96”, respectively.

[0084] Advantageously, querying the waste identifier database 526 enables waste products to be automatically and efficiently classified into one or more stock keeping units in real time. Furthermore, classification of the waste enables waste products be monitored and / or tracked, thereby providing useful data and decision intelligence regarding the waste. Additionally, updating the waste identifier database 526 in the manner described above enables the waste identifier database 526 to be iteratively improved upon. By processing waste 104 through one or more material processing facilities which may be distributed across a wide geographic area in accordance with the techniques described herein, large quantities of data regarding the waste 104 is produced. This large amount of data thereby contributes to more robust and accurate classification when the waste identifier database 526 is updated and subsequently queried for a waste unit classification. This is because the waste identifier database 526 comprises this large amount of data and is therefore more representative of waste as a whole when compared with a waste identifier database comprising a smaller amount of data.

[0085] The one or more outputs of the techniques described above (for example the object source scores 514 and / or the waste unit classifications 528) may be used as inputs to a control mechanism used to control one or more systems or processes. For example, the control mechanism may be used to control one or more mechanical, electrical, and / or electro-mechanical components of the material processing facility. For example, the one or more object source scores 514 and / or waste unit classifications 528 may be used to determine one or more instructions to be provided to the control mechanism, for example a controller. In one example, the one or more control mechanisms may control a conveyor belt carrying waste. In another example, the one or more control mechanisms may be an actuator, such as a robot configured to interact with the waste, for example by picking items of waste. The one or more object source scores 514 and / or the waste unit classifications 528 may be used as statistical data indicating the consumption of products in a given geographical area. In one example, the statistical data may also be used to determine the procurement of products. For example, the one or more outputs may be processed to determine that the waste received at a particular material processing facility includes a threshold amount of a particular waste unit class. This determination may be used to automate one or more further processes; such as order or manufacture more of a product corresponding to the waste unit class. For example, products corresponding to the waste unit class may include the product of the waste unit class itself or consumables related to manufacture or recycling of the waste unit class. Other uses of the statistical data may be envisaged. For example, the one or more outputs may be used for scheduling resource allocation or maintenance within the, or other, material processing facilities. For example, a determination that received waste includes a threshold amount of a particular waste unit class may be used to divert, allocate or obtain additional resources used for processing waste of the particular waste unit class.

[0086] As will be readily apparent, the methods and systems described herein enable the classification of waste products into one or more waste unit classifications 528 where the number of potential corresponding SKUs is very large. The methods and systems described herein achieve this in part by implementing a “passive analysis” approach while still providing for the option of manual review for detected objects (e.g. waste products) which may not correspond with an existing stock keeping unit in the waste identifier database 526. (e.g. the database update tool 116 of Figure 1 described above) Furthermore, because the methods and systems described herein are implemented in a distributed manner, (i.e. geographically distributed material processing facilities) this enables the wide distribution and variation of waste products in a given geographic area to be fully appreciated and incorporated into the analysis For example, the methods and systems described herein may operate in a first geographic area, which may not include a second geographic area. In this example, waste produced in the second geographic area may not be processed at a material processing facility, and therefore may not be fully appreciated and / or incorporated as data in the methods and systems described herein. This distribution can depend on such factors as seasonality, regulatory variations, localization and / or internationalization of consumer packaging, and / or production / supply chain / consumption rates. Additionally, the methods and systems described herein overcome a lack of public domain data by avoiding the need for a large and fully labelled waste specific training dataset by rather using the large amount of data made available as a consequence of the waste being processed though one or more material processing facilities. In such a way, stock keeping unit classification is provided for, and a large domain specific dataset may be generated and used in order to improve waste unit classification over time, which may not otherwise be possible using publically available datasets.

[0087] One alternative to the distributed implementation described herein is to use multiple localized models (e.g. one for each material processing facility 106) trained on data which has only been gathered in the specific geographic area in which that model operates (e.g. a material processing facility 106 which only uses data generated locally to train their respective machine learning models). There are a number of disadvantages to this approach. For example, the machine learning models described herein benefit from diversity of training data. In the localized implementation, the machine learning models lack diversity of training data because the only data they have available to them is the data generated from waste 104 received at their respective material processing facility 106 (e.g. waste 104 received from a single geographic area). Furthermore, it may be difficult to predict the composition of waste in a specific area, thereby making it difficult to maintain the localized implementation (e.g. the manual review of large amounts of waste from unknown provenance may be difficult). In addition, the composition of waste may change quickly in a specific area as opposed to the wider geographic scope of the distributed implementation requiring the use of additional mechanisms in order to recognize such an event. For example, multiple coffee shops may open in a specific area all within a short span of time, which may cause the composition of waste in that specific area to change rapidly over time. This would not have such a negative effect in the distributed implementation because of the larger geographic coverage provided by the distributed implementation. In general, the distributed implementation benefits from both the diversity of data provided by a wide geographic coverage and its overall ease of maintenance when compared with the localized implementation.

[0088] Any one of the material recognition model 504, the source determination model 510, the deduplication stage 512, the averaging module 520, and the waste identifier database 526 may be implemented on the edge or on the cloud. As described herein, the input image 502 is received as input on the edge. When the input image 502 is received as input at an edge device (i.e. a distributed computing device), the input image 502 is received by the material recognition model 504 (i.e. a portion of the waste classification pipeline 500). For example, in the case that the material recognition model 504 is implemented on the edge, the input image 502 may be input directly into the material recognition model 504. Alternatively, in the case that the material recognition model 504 is implemented on the cloud, the edge device may transmit the input image 502 to the cloud (e.g. one or more servers) over a network for processing by the material recognition model 504 on the cloud. In the case that a particular preceding portion of the waste classification pipeline 500 is implemented on the edge and where a proceeding portion of the waste classification pipeline 500 is implemented on the cloud, the edge device may transmit any intermediate data output from the preceding portion of the waste classification pipeline 500 to the cloud to be used by the proceeding portion of the waste classification pipeline 500. In the case that a particular preceding portion of the waste classification pipeline 500 is implemented on the cloud (e.g. one or more servers) and where a proceeding portion of the waste classification pipeline 500 is implemented on the edge, the cloud (e.g. one or more servers) may transmit any intermediate data output from the preceding portion of the waste classification pipeline 500 to the edge device to be used by the proceeding portion of the waste classification pipeline 500. Any combination of “on the cloud” and “on the edge” portions of the waste classification pipeline 500 may be envisaged. In other words, the waste classification pipeline 500 may be partitioned between a monitoring unit and one or more servers.

[0089] Figure 6 schematically depicts a diagram of a waste classification pipeline 600. The waste classification pipeline 600 may be configured to receive an input image 602 and output one or more waste unit classifications 616. The input image 602 may be received by the waste classification pipeline 600 as data representing the input image 602. The waste classification pipeline 600 may comprise a first model 604, a source determination model 608, and a waste identifier database 614. The first model 604 may be, for example, the material recognition model of Figure 3. The first model 604 may be configured to receive as input an input image 602 and output data indicating at least one object 606. The input image 602 may be the input image 602 may be the input image 302 402 502 of Figures 3, 4, and 5 respectively. The data indicating at least one object 606 may be one or more of the portion of the input image 406 described with reference to Figure 4, or may be any other data representing an object depicted in the input image 602. The first model 604 and the source determination model 608 may each be a machine learning model. The source determination model 608 may be configured to receive as input the data indicating at least one object 606 and output one or more object source scores 610 and / or one or more object feature vectors 612. In one example, the data indicating at least one object 606 may be one or more cropped images. For example, each cropped image may be the portion of the input image 406 described with reference to Figure 4. The one or more object source scores 610 may be data indicative that an object in the input image 602 originated or emanated from one or more sources. The object source scores 610 may be the object source scores 414 514 described with reference to Figures 4 or 5. The one or more object feature vectors 612 may be the object feature vectors 416 518 described with reference to Figures 4 and 5. The first model 604 and the source determination model 608 may each be a machine learning model, and may respectively be the material recognition model of Figure 3 and the source determination model of Figure 4.

[0090] The waste identifier database 614 may be configured to receive as input the one or more object source scores 610 and / or the one or more object feature vectors 612 and output the one or more waste unit classifications 616. The waste identifier database 614 may be the waste identifier database 526 described with reference to Figure 5. In particular, the waste identifier database 614 may output the one or more waste unit classifications 616 in response to receiving a query comprising the one or more object source scores 610 or the one or more object feature vectors 612, or both, as input. The waste unit classifications 616 may be classifications of one or more objects depicted in the input image 602. In one example, the waste unit classifications 616 may each correspond to one or more stock keeping units. For example, waste unit classifications 616 may indicate that a first object belongs to a “Metal drinks can” SKU and a “Drinks container” SKU, but not a “Plastic bottle” SKU. The same waste unit classifications 616 may also indicate that a second object depicted in the input image 602 may correspond to the “Plastic bottle” SKU and “Drinks container” SKU, but not the “Metal drinks can” SKU.

[0091] In general, the waste classification pipeline 600 is the waste classification pipeline 500 described with reference to Figure 5 without a number of non-essential components. For example, the deduplication stage 512 and the averaging module 520 may only be required in certain implementations. This may also apply to the object classifications 508 output by the material recognition model 504 of Figure 5. However, this does not infer that all of the components depicted in Figure 6 are essential to the invention. For example, the object source scores 610 and / or the object feature vectors 612 may be used together or separately to output the waste unit classifications 616. Alternatively, it may be desirable to retrieve the object source scores 610 and / or the object feature vectors 612 without querying the waste identifier database 614.

[0092] Each one of the material recognition model 300 of Figure 3, the source determination model 400 of Figure 4, the first model 604, and / or any other machine learning model described herein may be configured to substantially concurrently process multiple input images at any one time rather than only a single input image. The output of a machine learning model processing multiple input images substantially concurrently will be two or more of their respective output. For example, the input of three input images into a machine learning model configured to substantially concurrently process multiple input images may output three outputs.

[0093] As described herein, references to the term “on the edge” refers to computation, processing, and / or execution of a method on a distributed computing device (e.g. a monitoring unit). As described herein, references to the term “edge device” refers to a computing device in which the method referred to above is computed, processed, and / or executed. Advantageously, processing on the edge means that all waste is processed in real-time. Furthermore, bandwidth and / or latency limitations which may be associated with transmitting data to the cloud are not an issue with on the edge processing. As described herein, references to the term “on the cloud” refers to computation, processing, and / or execution of a method on a centrally located computing device (e.g. one or more servers). Advantageously, processing on the cloud means that images of waste transmitted to the cloud for processing may be stored for future historical analysis and / or other use. Furthermore, processing on the cloud enables the implementation of more computationally efficient and / or powerful computer hardware and / or methods used for processing which may not otherwise be available on the edge.

[0094] Figure 7 is a flow diagram depicting a waste unit classification method that may be carried out in accordance with the waste classification pipeline 202, 500, 600 described with reference to Figures 2, 5, and 6 respectively.

[0095] At step 700, waste is received at a material processing facility. The material processing facility may be any one of the material processing facilities 106 described with reference to Figure 1. The waste may comprise one or more waste objects. The waste may be the waste 210 described with reference to Figure 2, or any one of the waste 104 described with reference to Figure 1.

[0096] At step 702, data representing one or more images of the waste may be obtained by a waste monitoring unit of the material processing facility. The data representing one or more images of the waste may be image data, or any other suitable data. The data may be obtained by camera or any other suitable imaging device. With reference to Figure 2, the data may be obtained by the camera 204 imaging the waste 210.

[0097] At step 704, the data representing the one or more images is provided as input to a first model. The first model is configured to output, for at least one of the one or more object depicted in the waste, data indicating the at least one object in the one or more images. For example, the first model may be part of the waste classification pipeline 202, 500, 600. The data indicating the at least one object in the one or more images may be image data representing the one or more waste objects depicted in the one or more images. In other examples, the data indicating the at least one object in the one or more images may be latent data representing the images, such as an embedding generated from the image data. The first model may be the material recognition model 300 described with reference to Figure 3.

[0098] At step 706, an object source score for each of the at least one object depicted in the one or more images is output. In an example, the data indicating the at least one object is provided as input to a source determination model configured to provide a source determination model output indicating one or more corresponding object source scores. Each object source score is indicative of a source of the at least one object depicted in the one or more images. The source may be, for example, a type, class, specification, source of manufacture, method of manufacture, entity, and / or brand. The source determination model may be the source determination model 400 described with reference to Figure 4. The source determination model output may be data comprising one or more object source scores.

[0099] In one implementation, the source determination model output may comprise an object feature vector indicative of an object depicted in the one or more images. In this implementation, the object feature vector may be used to query a waste identifier database to determine one or more classifications of the waste each corresponding to a waste unit class in the waste identifier database. Each waste unit classification may correspond to one or more stock keeping units.

[0100] The methods and systems described herein have been described with reference to a material processing facility. The term material processing facility is intended to cover any facility that receives material, such as waste material, for further processing. For example, the methods and systems may be used in recycling plants, material recovery facilities, composting facilities, anaerobic digestion facilities, paper and pulp mills, and / or any other suitable type of facility.

[0101] Figure 8 schematically illustrates an exemplary arrangement of components which may provide a computing system 4 used to implement all or part the waste classification pipeline 500, the monitoring units 108, and the one or more servers 206.

[0102] A processor, in this case in the form of a CPU 4a, configured to read and execute instructions stored in a volatile memory 4b which takes the form of a random access memory. It will be appreciated that the processor may take other forms, such as, for example, a GPU. The volatile memory 4b stores instructions for execution by the CPU 4a and data used by those instructions. For example, the instructions may include instructions for causing the the waste classification pipeline 500, the monitoring units 108, and the one or more servers 206 with reference to any of the appended Figures.

[0103] The computing system 4 comprises a storage device 5. It will be appreciated that the storage device 5 may be implemented in any way, such as for example, a hard disk drive, a solid state drive, etc. The computing system 4 further comprises an I / O interface 4d to which are connected peripheral devices used in connection with the computing system. More particularly, a display 4e is configured so as to display output. Input devices are also connected to the I / O interface 4d. Such input devices include a keyboard 4f and a mouse 4g which allow user interaction with the computing system 4. A network interface 4h allows the computing system 4 to be connected to appropriate computer networks, such as the Internet 6, and so as to be able to send and receive from and to other computing devices such as computing devices of the users (e.g. where the computing system 4 provides the waste classification pipeline 500, the monitoring units 108, and the one or more servers 206).206.The CPU 4a, volatile memory 4b, the storage device 5, I / O interface 4d, and network interface 4h, are connected together by a bus 4i.

[0104] The techniques described above may be implemented in hardware, firmware, software, or any combination thereof. The techniques may also be implemented as instructions stored on a machine-readable medium, which may be read and executed by one or more processors. A machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device). For example, a machine-readable medium may include read only memory (ROM); random access memory (RAM); magnetic storage 5 media; optical storage media; flash memory devices; electrical, optical, acoustical or other forms of propagated signals (e.g. carrier waves, infrared signals, digital signals, etc.), and others. Further, firmware, software, routines, instructions may be described herein as performing certain actions. However, it should be appreciated that such descriptions are merely for convenience and that such actions in fact result from 10 computing devices, processors, controllers, or other devices executing the firmware, software, routines, instructions, etc. and in doing that may cause actuators or other devices to interact with the physical world.

[0105] While specific embodiments of the invention have been described above, it will be appreciated that the invention may be practiced otherwise than as 15 described. The descriptions above are intended to be illustrative, not limiting. Thus it will be apparent to one skilled in the art that modifications may be made to the invention as described without departing from the spirit of the invention.

Claims

1. A computer-implemented method for monitoring waste at one or more material processing facilities each comprising one or more waste monitoring units, the method comprising:receiving waste at a material processing facility, the waste comprising one or more objects;obtaining, by a waste monitoring unit of the material processing facility, one or more images of the waste;providing data representing the one or more images as input to a first model configured to output, for at least one of the one or more objects depicted in the one or more images, data indicating the at least one object; andoutputting an object source score for each of the at least one object, comprising:providing the data indicating the at least one object as input to a source determination model configured to output a source determination model output indicating one or more corresponding object source scores, each object source score indicative of a likelihood that a source of one of the at least one objects depicted in the one or more images can be determined.

2. The computer-implemented method of claim 1, wherein method further comprises determining, based upon the data indicating the at least one object, one or more object feature vectors indicative of the at least one object depicted in the one or more images.

3. The computer-implemented method of claim 1 or 2, further comprising:querying, based upon the one or more object source scores and / or the one or more object feature vectors, a waste identifier database to determine one or more corresponding waste unit classifications for each of the at least one object, the waste unit classifications each corresponding to one of one or more waste unit classes in the waste identifier database.

4. The computer-implemented method of claim 3, wherein at least one of the at least one waste unit classification corresponds to one or more stock keeping units.

5. The computer-implemented method of any preceding claim, wherein the first model is a material recognition model configured to output one or more object material classifications.

6. The computer-implemented method of any preceding claim, wherein the first model comprises:a feature extraction model configured to receive as input the data representing the one or more images and to output an image-level feature map comprising data indicative of one or more features corresponding to the one or more images;an object detector configured to receive as input the feature map and to output the data indicating the at least one object in the one or more images; andan object classifier configured to receive as input the feature map and to output one or more object classifications.

7. The computer-implemented method of claims 2 to 6, wherein the source determination model comprises:an object feature extraction model configured to receive as input the data indicating the at least one object in the one or more images and to output an objectlevel feature map;one or more source classification layers, the one or more source classification layers connected to the object feature extraction model to receive as input the objectlevel feature map and configured to output the source determination model output; andone or more feature layers connected to the object feature extraction model to receive as input the object-level feature map and configured to output the one or more object feature vectors.

8. The computer-implemented method of any preceding claim, wherein the source is a type, class, specification, source of manufacture, method of manufacture, entity, and / or brand.

9. The computer-implemented method of claims 3 to 8, the method further comprising:updating, based upon the one or more object feature vectors and their corresponding waste unit classifications, the waste identifier database.

10. The computer-implemented method of any preceding claim, further comprising:for one of the one or more objects depicted in the one or more images, determining, based upon the corresponding object source score, that an object source cannot be determined.

11. The computer-implemented method of any preceding claim, further comprising:updating, based upon candidate data comprising one or more candidate object source scores and / or one or more candidate object feature vectors, one or more waste unit classes stored in a waste identifier database.

12. The computer-implemented method of claim 11, wherein updating the waste identifier database comprises:displaying an image of a candidate object on a display of a user interface, the candidate object corresponding to one of the one or more candidate object source scores and / or one of the one or more candidate object feature vectors;receiving, from a user by the user interface, an indication of one or more waste unit classifications for the candidate object, each waste unit classification corresponding to one of the one or more waste unit classes in the waste identifier database; andassigning, based on the indication, the candidate object to the one or more waste unit classes indicated by the classification indication.

13. The computer-implemented method of claim 3 or any claim dependent thereon, wherein querying the waste identifier database further comprises:determining a similarity value between each object feature vector and one or more waste unit cluster centroids, each waste unit cluster centroid corresponding to a different one of the one or more waste unit classes; anddetermining the waste unit classifications based upon whether the one or more similarity values meet a similarity criterion.

14. The computer-implemented method of claim 13, wherein the one or more similarity values are cosine similarity values.

15. The computer-implemented method of any preceding claim, the method further comprising:controlling, based upon the source determination model output and / or the one or more waste unit classifications, one or more mechanical, electrical, and / or electromechanical control mechanisms of the material processing facility.

16. The computer-implemented method of any preceding claim, the method further comprising:outputting, based upon the source determination model output and / or the one or more waste unit classifications, output data indicative of waste production in one or more geographic areas.

17. The computer-implemented method of any preceding claim, wherein the one or more material processing facilities comprises a plurality of material processing facilities each located in a different geographic area;the method further comprising:determining an update to the source determination model and / or the firstmodel; andupdating local instances of the source determination model and / or first model at each of the plurality of material processing facilities.

18. The computer-implemented method of any preceding claim, further comprising:determining that a plurality of images obtained by the waste monitoring unit each include a respective depiction of a same object;providing data representing each of the respective depictions to the first model to output data indicating the same object; andwherein outputting the one or more object source scores for the at least one object further comprises:providing the data indicating the same object to the source determination model;outputting a respective source determination model output for each depiction of the same object; anddetermining a single object source score for the same object based on the plurality of respective source determination model outputs.

19. The computer-implemented method of claim 18, wherein determining the object source score for the same object comprises averaging the plurality of respective source determination model outputs.

20. The computer-implemented method of claim 2 or any claim dependent thereon, comprising:determining that a plurality of images obtained by the waste monitoring unit each include a respective depiction of a same object;providing data representing each of the respective depictions to the first model to output data indicating the same object; andwherein determining the one or more object feature vectors indicative of the at least one object further comprises:determining, based upon the data indicating the same object, a respective object feature vector for each depiction of the same object; anddetermining an object feature vector for the same object based on the plurality of respective object feature vectors.

21. The computer-implemented method of claim 20, wherein determining the object feature vector for the same object comprises averaging the plurality of respective object feature vectors.5     22. A system comprising:one or more servers comprising a waste identifier database and one or more processors; andone or more material processing facilities each configured to receive waste and each comprising one or more monitoring units, the monitoring units each comprising 10 one or more cameras and one or more processors, each monitoring unit being configured to carry out the method of any of claims 1 to 21.

23. A computer readable storage medium comprising computer readable instructions, the computer readable instructions, when executed by a processor, cause 15 the processor to carry out the computer-implemented method of any of claims 1 to 21.

Images