Inspecting items in a stream of items by means of a classifiers

Abstract

The present disclosure relates to a method of inspecting items at a feeding arrangement (1), and a computer readable storage medium having stored thereon instructions for implementing the method. The method comprises: at a feeding arrangement (1): feeding (S1') a stream of items through an inspection zone; capturing (S2') a set of images of the items when in the inspection zone; providing (S3') a dataset (100) comprising item representations (101, 102, 103, 104, 105) and labels, providing (S4') at least one classifier configured to classify at least one image of said set of images based on a similarity metric between said at least one image and at least one item representation (101, 102, 103, 104, 105) of said dataset (100), classifying (S5') at least one image using at least one classifier of said at least one classifier and the dataset (100); associating (S6') each item representation (101, 102, 103, 104, 105) and / or label in said dataset (100) with a respective timestamp, and updating (S7') the respective timestamps whenever an associated item representation (101, 102, 103, 104, 105) is identified as a match by at least one of said at least one classifier, and updating (S8') the dataset by removing any item representations (101, 102, 103, 104, 105) and / or labels having an associated timestamp indicating a predetermined time period has lapsed since said item representation (101, 102, 103, 104, 105) was last identified by the classifier to be a match.

Description

[0001] INSPECTING ITEMS IN A STREAM OF ITEMS BY MEANS OF A CLASSIFIERS

[0002] Technical field

[0003] The present inventive concept relates, in general, to a method of inspecting items in a stream of items.

[0004] Background

[0005] Inspecting a stream of items, such as in manufacturing, mineral processing, waste sorting, or recycling, can be automated using classifiers to identify and categorize items. These classifiers are typically machine learning models trained on datasets that represent the expected items. However, when there is a large diversity of items in the stream or a wide variety of any given item type, the complexity of the classifier increases. This increased model complexity can slow down processing times, reduce system throughput, and heighten the risk of blockages and / or stops in the feeding of the stream. Additionally, when new items appear, the classifier may require more time for recognition, further delaying the process.

[0006] In light of these challenges, there is a need for an improved method of inspecting items in a stream of items.

[0007] Summary

[0008] To alleviate at least one of the above drawbacks and / or achieve at least one of the above objects and optionally also other objects and advantages that will be evident from the following description, a method having the features defined in claim 1 is provided according to the present inventive concept. Preferred variants will be evident from the dependent claims.

[0009] According to a first aspect of the present invention, a method of inspecting items in a feeding arrangement is provided, the method comprising: at a feeding arrangement: feeding a stream of items through an inspection zone; capturing a set of images of the items when in the inspection zone; providing a dataset comprising item representations and labels, providing at least one classifier configured to classify at least one image of said set of images based on a similarity metric between said at least one image and at least one item representation of said dataset, classifying at least one image using at least one classifier of said at least one classifier and the dataset; associating each item representation and / or label in said dataset with a respective timestamp, and updating the respective timestamps whenever an associated item representation is identified as a match by at least one of said at least one classifier, and updating the dataset by removing any item representations and / or labels having an associated timestamp indicating a predetermined time period has lapsed since said item representation was last identified by the classifier to be a match.

[0010] Hereby, a method which automates and simplifies the inspecting of items in a stream of items is provided. Further, the dataset, which the at least one classifier uses to classify items in the stream of items, may be updated over time to exclude item representations and / or labels that correspond to items that seemingly no longer occur in the stream of items as of late. Thereby, the dataset may advantageously be optimized over time to comprise only relevant item representations for the stream of items being inspected. The dataset may as a consequence occupy less storage space on a storage medium storing the dataset. Further, the storage space on a storage medium storing the dataset may be used more efficiently. Further, with a decreased size of the dataset, the classification speed of the at least one classifier may also increase, as it requires less time and computational resources to determine the similarity metric between an item representation of an inspected item and each item representation in the dataset. By increasing the speed of the at least one classifier, throughput of the feeding arrangement may also be increased, and consequently a risk of blockage is reduced and / or stops in the feeding of the stream is reduced.

[0011] The process of excluding item representations that correspond to items that has not occurred, or has at least not been identified, in the stream of items for a predetermined time may be referred to as a dementia process. Thus, a processing unit (or controller or control device) implementing the at least one classifier may be configured to implement the dementia process to optimize the dataset said at least one classifier uses when classifying items in the stream of items.

[0012] As mentioned above, the method comprises associating each item representation and / or label in said dataset with a respective timestamp. Here, a timestamp may be a record of time at which a particular event occurred, e.g., when an image was captured or when an image was introduced in the dataset, or when a label was established, a timestamp may be indicative of the peak-time of the item, i.e. a time when many such items occurred, or may be indicative of any other time relevant for this context. Additionally, or alternatively, a time stamp may comprise a time in the future to prevent the item representation to be deleted before a date set in the future. The time stamp may comprise a date component, e.g., a date of time at which the particular event occurred. Optionally, the time stamp may comprise a time component, e.g., the time of day at which the particular event occurred (i.e., specified in hours, minutes, and optionally second or fractions thereof). Optionally, the time stamp may comprise time zone information. Optionally, the time stamp may reflect a local time reference or be synchronized to a universal time standard.

[0013] The method of inspecting described above may generally be applied to any type of stream of items. Stream of items may be provided in various applications such as waste sorting, manufacturing, or recycling.

[0014] A stream of items can undergo changes in terms of composition depending on the season. These changes can affect the types, quantities, and characteristics of items in the stream. The method of inspecting described herein, which implements the aforementioned dementia process, may advantageously update the dataset to reflect these seasonal changes. Thus, speed of the at least one classifier may advantageously be enhanced.

[0015] Further, the classification of the items in a stream of items may also change depending on changes in legislation and / or policies. For instance, some items may be reclassified as hazardous items, or some hazardous items may be reclassified as non-hazardous items. By updating the dataset to reflect these legislation and / or policy changes, the accuracy and / or speed of the at least one classifier may advantageously be enhanced over time.

[0016] The dataset may comprise a number of different item representations and labels for the same item type, but repeated classification by the at least one classifier indicates that at least one item representation is redundant or less frequently identified as a match contrary to an ad hoc assumption, i.e. , the item representation is a poor item representation, thus indicating that the dataset may be reduced without sacrificing classification capability or accuracy thereof of the at least one classifier. By updating the dataset to reduce redundancy or remove poor item representations or remove items no longer of interest, speed of the at least one classifier may advantageously be enhanced. This update may e.g. be executed by removing, from a group of image representation and / or labels all having the same time stamp, the image representation and / or labels that have the oldest timestamp(s).

[0017] A feeding arrangement is an arrangement used to deliver a stream of items passed an inspection zone. The feeding arrangement may ensure that items are presented in a consistent and manageable way for further inspection, classification, processing, and / or sorting.

[0018] A processing plant (such as a sorting plant) may comprise said feeding arrangement.

[0019] According to one embodiment, said classifying at least one image using the at least one classifier of said at least one classifier and the dataset comprises: identifying among said item representations of said data set at least one candidate item representation based on said similarity metric between said at least one image and at least one item representation of said dataset; determining at least one matching item representation of said at least one candidate item representations based on a predetermined matching condition; and associating said at least one image with one or more labels associated with said at least one matching item representation.

[0020] Said at least one candidate item representation may comprise one candidate item representation. Said at least one candidate item representation may according to a non-limiting example comprise a plurality of candidate item representations, such as 2, 3, 4, 5, 5-10, 10 or more item representations.

[0021] Said candidate item representation may be provided by applying the similarity metric between the at least one image and the at least one item representations, or a plurality of item representations, of the dataset.

[0022] At least one matching item representation may be determined among said at least one candidate item representations based on a predetermined matching condition. It should be understood that "matching” may generally refer to the most similar or a number of most similar item representations among the item representations in the dataset. In other words, matching does not necessarily mean that identical.

[0023] According to one embodiment, said predetermined matching condition comprises at least one of: nearest neighbor approach, nearest plurality of neighbor approach, or histogram threshold approach.

[0024] As a non-limiting example, nearest neighbor approach may, depending on the item representation and / or said metric, be implemented as a closest distance approach. In nearest plurality of neighbor approach, the same principle as in nearest neighbor approach may be implemented, but instead of finding the nearest, a plurality of the nearest neighbors is provided, e.g., the five nearest neighbors. In histogram threshold approach, a distribution of similarity scores is evaluated using a threshold to determine whether an image should be assigned a label or remain unclassified. The present invention is however not limited to any of these matching conditions, but may implement other matching conditions.

[0025] According to one embodiment, the feeding arrangement comprises at least one of a chute, a conveyer belt, and a free falling path, wherein said inspection zone is provided at one of said at least one of a chute, a conveyer belt, and free falling path. The feeding arrangement may comprise any one of these elements, and any number of these elements, to provide a stream of item from a first location A to at least a second location B. The feeding arrangement may further comprise a feed regulator configured to control a rate at which items are introduced in the stream of items. A feed regulator may be a gate or a funnel.

[0026] The feeding arrangement may comprise a camera arrangement, such as but not limited to: a visible spectrum (VIS) camera; a Near-Infrared (NIR) camera; a thermal infrared camera; a spectrometer with image functionality (such as Specim FX10, FX15, or FX50); a hyperspectral camera; a multispectral camera; an ultraviolet (UV) camera; a 3D camera (e.g., a structured light camera, a time-of-flight camera, a stereo vision camera); a line camera; and a matrix camera. The camera arrangement may capture said set of images of the items when in the inspection zone.

[0027] The set of images of items in the stream of items are captured when items are in an inspection zone. Here, images may be two-dimensional, 2D, representations of image data provided by a camera arrangement like any of the ones listed above. Thus, said capturing a set of images of items refers to a camera arrangement providing image data representing at least one item when it was in the inspection zone. Image data may be any data from which a 2D representation of an item may be provided. An item representation, however, is not strictly limited to being an image, i.e. , a 2D representation, but may be any viable and / or suitable representation, based on said images or said image data. Further, said set of images may comprise one or more images, such as 10 images, or 100 images, or 1000 images, or 10000 images. More than one image may be taken of any one item. Consequently, an item may be present in more than one image. Further, more than one item may be present in any one image.

[0028] Images may be taken by any type of camera arrangement known in the art, e.g., for instance any type of camera arrangement exemplified above.

[0029] The item representations of the dataset may each be labeled by a corresponding label of said labels.

[0030] The at least one classifier may be trained on a labeled set of item representations (e.g., labeled images). The at least one classifier may be configured to identify a depicted item in an inspection image (or an item representation based on said inspection image). Images of said set of images may be referred to as inspection images. The at least one classifier may be configured to determine a class or type of the item depicted in an image or represented by said item representation. For example, identifying a depicted item may comprise determining that the item is of the type “pants”, or “shirts”, or “red pants”, or “blue shirt”, or “can”, or “aluminum can”, etc. Further, it is to be understood that an inspection image may depict more than one item. Thus, labeling the inspection images may comprise identifying, for any one image, more than one item.

[0031] Identifying a depicted item may be performed in a variety of ways. For example, identifying the depicted item may be performed by foregroundbackground segmentation. Alternatively, identifying the depicted item may be performed by object instance segmentation.

[0032] Object instance segmentation is particularly beneficial if any items in the stream of items are stacked, since it may be used in order to distinguish between individual items in a stack of items.

[0033] Object instance segmentation may be performed by a machine learning model. For example, the machine learning model may be part of or be the same machine learning model as the classifier. As such, object instance segmentation may be performed by a neural network, such as a convolution neural network or a fully connected neural network. After an inspection image has been processed by a machine learning model using object instance segmentation, it may be provided to the classifier for classification.

[0034] Images of the set of inspection images may be subject to further image processing steps. For example, images of the set of inspection images may be subject to image improvement, cropping, etc. Furthermore, it is to be understood that a captured inspection image may be discarded from the set, for example if no object can be identified, or if the inspection image is of too low quality.

[0035] The method may further comprise: extracting, from the set of inspection images, a set of inspection item images, each inspection item image depicting one identified item such that the number of inspection item images in the set of inspection item images may be identical to a number of items identified in the set of inspection images. Alternatively, the method may comprise: extracting, from the set of inspection images, a set of inspection item representations, each inspection item representation representing one identified item such that the number of inspection item representations in the set of inspection item representations may be identical to a number of items identified in the set of inspection images.

[0036] Hereby, if an inspection image depicts more than one item, a set of inspection item images (or item representations) may be extracted from the inspection image, thereby producing a cleaner inspection data to be classified by the at least one classifier. Further, an inspection item image may have a smaller data size than the original inspection image. Hereby, a set of inspection images which is easier to process is provided.

[0037] According to one non-limiting embodiment, said stream of items comprises any one of, or any combination of: minerals, food, waste, clothing, and recyclables. Minerals may refer to solid, naturally occurring inorganic substances found in the Earth's crust, such as rocks, ores, and gemstones. Food may refer to edible items or products derived from plants, animals, or synthetic processes. Waste may refer to unwanted or discarded materials, typically generated by households, industries, or commercial activities. Clothing may refer to textiles or garments made from materials such as cotton, polyester, wool, or blends. Recyclables may refer to items that can be processed and re-used in the production of new products, such as PET bottles.

[0038] According to one embodiment, said feeding arrangement comprises at least one sorting arrangement configured to sort said stream of items into at least one sorted fraction and optionally at least one unsorted fraction, the method comprising: at the at least one sorting arrangement: controlling sorting of items in the stream of items into said at least one sorted fraction and optionally said at least one unsorted fraction using said set of images and said at least one classifier. A sorting process of the sorting arrangement may be controllable via a controller configured to control said sorting process. Said at least one classifier may be implemented on said controller. The controller may comprise a non-transitory storage medium having stored thereon instructions for implementing the at least one classifier. The controller may comprise a processing unit configured to execute the instructions on said non-transitory storage medium.

[0039] According to one embodiment, said at least one classifier is a machine learning model trained on item representations and labels.

[0040] Labels may refer to categories or classes that are assigned to an item representation based on its content. Labels may provide a ground truth or may serve as a correct or at least predetermined classification for a given item representation. Each item representation may be associated with at least one of said labels. As a non-limiting example, an item representation may be a data representation of a particular item, such as a pair of jeans, and the item representation of said pair of jeans may be associated with a label indicating “blue pair of jeans” and / or “size small” and / or “boot cut pair of jeans”. The dataset may comprise many different variations of the same general type of item. A label may, e.g., be a keyword, such as “green”, or “pants”, or “aluminum.” An item representation may have more than one label attached or related to it. A label may be a multiclass label, i.e. a label containing more than one element. That is, a label may be a vector containing a plurality of elements. For example, a label may be a multiclass label containing the elements of “pants”, “short”, and “green”.

[0041] According to one embodiment, said at least one classifier is a machine learning model trained on the item representations and the labels of said dataset. The machine learning model may be any type of machine learning model. For example, the machine learning model may be a neural network, such as a convolution neural network or a fully connected neural network. The machine learning model may be a neural network having any number of input nodes, any number of hidden layers and respective nodes in such layers, and any number of output nodes. Further, any node of the neural network may have any type of activation function, such as the rectified linear unit (ReLU) activation function, or the logistic function. According to one embodiment, said item representations are images, or feature representations provided based on images. Item representations may however be any other viable and / or suitable item representation, such as a histogram or a vector representation.

[0042] According to one embodiment, said similarity metric is any one of: a distance-based metric, a histogram-based metric, a statistical metric, a structural or visual similarity metric, and a neural network-learned metric. The inspected image may be provided as an item representation in a viable form of any of these similarity metrics, and the dataset may comprise item representations of the same form. The at least one classifier determines the similarity metric between the item representation of the item being inspected and the item representations in the dataset.

[0043] Distance-based metric may refer to a metric which measures the similarity between two item representations based on distance between their feature vectors in a given space. As non-limiting examples, distance-based metric may be any one of: Euclidian distance; Manhattan Distance; Cosine Similarity; and Mahalanobis Distance.

[0044] Histogram-based metric may refer to a metric which measures the similarity between two images based on the distribution of pixel values (such as color or intensity). As non-limiting examples, histogram-based metric may be any one of: color histogram; grayscale histogram; Chi-Square Distance; and Intersection of histograms.

[0045] Statistical metric may refer to a metric which uses statistical properties of the data to quantify the similarity between two items, e.g., by comparing distributions or moments. As non-limiting examples, statistical metric may be any one of: Mean Squared Error (MSE); Peak Signal-to-Noise Ratio, PSNR); Kullblack-Leibler, KL, Divergence; and Correlation Coefficient.

[0046] Structural or Visual Similarity metric may refer to a metric which measures the similarity based on the structural or visual features of an image, emphasizing aspects like texture, edges, and patterns. As non-limiting examples, a structural or visual similarity metric may be any one of: Structural Similarity Index, SSIM; Edge Detection; and Phase Congruency. Neural network learned metric may refer to a metric which is a similarity measure learned by a (deep) neural network through training on large datasets, allowing the model to capture complex, non-linear relationships between items. As non-limiting examples, a neural network learned metric may be provided based on any one of Convolutional Neural Networks, CNNs; Triplet Loss; and Siamese Networks.

[0047] According to one embodiment, the method further comprising distributing the at least one classifier from the first feeding arrangement to a second feeding arrangement via a network infrastructure, such as a cloudbased server, or via a non-transitory memory storage transfer process. The network infrastructure may be an ethernet-based infrastructure. The classifier may at the first sorting arrangement be stored on a non-transitory storage medium, which medium is subsequently transported to the second sorting arrangement. The distribution of the at least one classifier may be automated.

[0048] According to one embodiment, the feeding arrangement comprises at least a first sorting arrangement configured to sort said stream of items into at least one sorted fraction and optionally at least one unsorted fraction, said inspection zone is provided at said first sorting arrangement, the method comprising: at a first sorting arrangement: sorting said stream of items into said at least one sorted fraction and optionally said at least one unsorted fraction; wherein said capturing a set of images comprises capturing a set of fraction images of the items in the at least one sorted fraction when in said inspection zone of said sorting arrangement; labeling the fraction images captured at the first sorting arrangement with fraction labels, wherein each fraction label is indicative of the sorted fraction of the at least one sorted fraction into which a depicted item has been sorted; wherein said providing said dataset comprises providing a dataset comprising at least item representations based on said fraction images and labels based on said fraction labels, wherein said providing at least one classifier comprises constructing said least one classifier configured to correlate items with fractions, the at least one classifier being a machine learning model and wherein constructing the at least one classifier comprises training the machine learning model using the fraction images captured at the first sorting arrangement and the fraction labels.

[0049] According to one embodiment, the feeding arrangement comprises a second sorting arrangement arranged downstream to the first sorting arrangement, the method comprising: at the second sorting arrangement: capturing images of items in a stream of items; controlling sorting of items in the stream of items into at least one sorted fraction and optionally at least one unsorted fraction using the images captured at the second sorting arrangement and the at least one classifier.

[0050] Hereby, a method which automates and simplifies sorting processes at multiple sorting arrangement is provided. Since the classifier is constructed solely by observation of the at least one sorted fraction, without necessitating any information about the internal operations of any actual sorting step, a more cost efficient and time efficient solution is provided.

[0051] The method may be used for any type of sorting processes. That is, the method may be implemented for any type of sorting arrangements and any type of item streams. A stream of items may thus comprise any type of items or any combination of items, such as clothing, minerals, foodstuff, empty containers, such as metal cans, etc. That is, the items in the stream of items may be made of or comprise plastics, metal, textile material, organic material, mineral material, etc.

[0052] Within the context of the present disclosure, the term “Sorting arrangement” is therefore to be understood as a system in which sorting processes are carried out located at a specific location. A sorting arrangement may, e.g., comprise a conveyor belt transporting a stream of items at which operators carrying out a manual sorting process are situated. Alternatively, a sorting arrangement may comprise a conveyor belt transporting a stream of items being monitored by an automated sorting setup. Thus, a first and a second sorting arrangement are to be understood as being spatially separated, i.e. sorting different streams of items. A first sorting arrangement may be arranged at a first sorting plant, whereas a second sorting arrangement may be arranged at a second sorting plant. Alternatively, a first and a second sorting arrangement may be arranged at the same sorting plant. The method may thus be implemented for sorting arrangements located at different parts of a plant, different parts of a city, different parts of a country, in different countries, and / or on different continents.

[0053] At the first and / or the second sorting arrangement, the respective stream of items may be sorted into one or more sorted fractions. The respective stream of items may e.g. be sorted into a first and a second sorted fraction. Alternatively, the stream of items may be sorted into different numbers of sorted fractions. That is, the stream of items at the first sorting arrangement may be sorted into two sorted fractions, whereas the stream of items at the second sorting arrangement may be sorted into one sorted fraction. That is, even if the classifier is constructed using two sorted fractions, the classifier may be distributed to and implemented in another sorting arrangement for only sorting a stream of items into one sorted fraction.

[0054] Within the context of the present disclosure, the term “Unsorted fraction” is to be understood as a fraction the contents of which is not purposively selected. That is, the at least one unsorted fraction comprises items which were not sorted into the at least one sorted fraction. The stream of items may, e.g., be clothing comprising both pants and shirts of different colors, wherein the sorted fraction consists of red shirts. Consequently, the unsorted fraction comprises shirts of colors other than red, as well as pants of different colors. Hence, such an unsorted fraction may constitute items which are to be discarded from the sorting process. Additionally or alternatively, an unsorted fraction may be recirculated into the sorting arrangement, or be provided to any other sorting arrangement.

[0055] In an exemplary sorting arrangement, a stream of items is provided in the form of a stream of clothing comprising pants of different colors, wherein pants of a first color are sorted into a first sorted fraction, whereas pants of a second color are sorted into a second sorted fraction. Optionally, clothing of any other color than the first and second color will be sorted into an unsorted fraction. For example, such unsorted fraction may be provided to another sorting arrangement for sorting into sorted fractions.

[0056] In another exemplary sorting arrangement, a stream of items is provided in the form of a stream of clothing of different types, wherein pants are sorted into a first sorted fraction, whereas shirts are sorted into a second sorted fraction. Optionally, clothing of any other type than pants and shirts may be sorted into an unsorted fraction. For example, such unsorted fraction may be provided to another sorting arrangement for sorting into sorted fractions.

[0057] In yet another exemplary sorting arrangement, a stream of items is provided in the form of objects made of different materials, wherein plastic objects are sorted into a first sorted fraction, whereas metal objects are sorted into a second sorted fraction. Optionally, objects made of any other material may be sorted into an unsorted fraction. For example, such unsorted fraction may be provided to another sorting arrangement for sorting into sorted fractions, or be discarded entirely by, e.g., dumping or incineration.

[0058] That an item is “depicted” is within the context of the present disclosure not be taken as meaning that the item must have been imaged by detecting radiation within visible wavelength bands. That is, “depicting” an item may be performed with radiation of any wavelength, such as infrared, ultraviolet, X- rays, etc.

[0059] Capturing a set of fraction images of the items in the at least one sorted fraction may be performed at any point after the items have been separated from the stream of items. For example, if the sorting arrangement comprises a conveyor belt for providing the stream of items to the location for physical sorting, wherein after such physical sorting step, each sorted fraction is transported on a separate conveyor belt, a set of fraction images may be captured at any instance at which the items of the sorted fraction are present on such separate conveyor belt. As another example, a set of fraction images may be captured when the items in a sorted fraction have been separated from the stream of items, but are stationary. It is to be understood that a set of fraction images may comprise any non-zero number of images. For example, a set of fraction images may comprise one or more images, such as 10 images, or 100 images, or 1000 images, or 10000 images.

[0060] More than one fraction image may be taken of any one item. Consequently, an item may be present in more than one image. Further, more than one item may be present in any one figure. For the purpose of the present inventive concept, it is not necessary to be able to keep track of any one object. Rather, the purpose of the set of fraction images is to train the classifier, the classifier being a machine learning model.

[0061] Images may be taken by any type of camera arrangement known in the art.

[0062] The machine learning model may be any type of machine learning model. For example, the machine learning model may be a neural network, such as a convolution neural network or a fully connected neural network. The machine learning model may be a neural network having any number of input nodes, any number of hidden layers and respective nodes in such layers, and any number of output nodes. Further, any node of the neural network may have any type of activation function, such as the rectified linear unit (ReLU) activation function, or the logistic function.

[0063] As is well known in the art, an image label is to be understood as information attached or related to an image and pertaining to any feature depicted by the image, or any context in which the image was taken, etc., that may be of importance for training a machine learning model. Here, a label is indicative of the sorted fraction of the at least one sorted fraction into which a depicted item has been sorted. That is, a label may, e.g., be a keyword, such as “green”, or “pants”, or “aluminum.” An image may have more than one label attached or related to it. A label may be a multiclass label, i.e. a label containing more than one element. That is, a label may be a vector containing a plurality of elements. For example, a label may be a multiclass label containing the elements of “pants”, “short”, and “green”. Labelling the fraction images may further comprise: for each fraction image: identifying at least one depicted item, wherein the fraction label is further indicative of each identified item.

[0064] Identifying a depicted item may, e.g., comprise determining a class or type of the item. For example, identifying a depicted item may comprise determining that the item is of the type “pants”, or “shirts”, or “red pants”, or “blue shirt”, or “can”, or “aluminum can”, etc. Further, it is to be understood that a fraction image may depict more than one item. Thus, labeling the fraction images may comprise identifying, for any one image, more than one item.

[0065] Identifying a depicted item may be performed in a variety of ways. For example, identifying the depicted item may be performed by foregroundbackground segmentation. Alternatively, Identifying the depicted item may be performed by object instance segmentation.

[0066] Object instance segmentation is particularly beneficial if any items in the stream of items are stacked, since it may be used in order to distinguish between individual items in a stack of items.

[0067] Object instance segmentation may be performed by a machine learning model. For example, the machine learning model may be part of or be the same machine learning model as the classifier. As such, object instance segmentation may be performed by a neural network, such as a convolution neural network or a fully connected neural network. After an image has been processed by a machine learning model using object instance segmentation, it may be provided to the classifier as training material.

[0068] Images of the set of fraction images may be subject to further image processing steps. For example, images of the set of fraction images may be subject to image improvement, cropping, etc. Furthermore, it is to be understood that a captured fraction image may be discarded from the set, for example if no object can be identified, or if the image is of too low quality.

[0069] The method may further comprise: extracting, from the set of fraction images, a set of fraction item images, each fraction item image depicting one identified item such that the number of fraction item images in the set of fraction item images may be identical to a number of items identified in the set of fraction images; and wherein labeling may be performed on the set of fraction item images, and wherein training the machine learning model may be performed using the fraction item images captured at the first sorting arrangement and the fraction labels.

[0070] Hereby, if an image depicts more than one items, a set of item images may be extracted from the image, thereby producing a cleaner training dataset. Further, a fraction item image may have a smaller data size than the original fraction image. Hereby, a set of images which is easier to process is provided.

[0071] It should be noted that any one item image of the set of fraction item images may be discarded.

[0072] The method may further comprise: at the first sorting arrangement, capturing a set of stream images of the items in the stream of items before sorting the stream of items into at least one sorted fractions and optionally at least one unsorted fraction, and labeling the stream images with stream labels, wherein labeling may comprise, for each stream image, identifying at least one depicted item, wherein the stream label may further be indicative of each identified item.

[0073] It is to be understood that a set of stream images may comprise any non-zero number of images. For example, a set of stream images may comprise one or more images, such as 10 images, or 100 images, or 1000, or 10000 images.

[0074] The set of stream images may be processed in accordance with what has been described with regards to the set of fraction images above. Specifically, images of the set of stream images may be subject to further image processing steps. For example, images of the set of stream images may be subject to image improvement, cropping, etc. Furthermore, it is to be understood that a captured stream image may be discarded from the set, for example if no object can be identified, or if the image is of too low quality. Since a stream image may depict a first and a second item that substantially differ from each other in terms of classification, i.e. “blue jeans” and “green shirt”, or “aluminum can” and “plastic bottle”, a stream image may be provided with more than one label, or a multiclass label

[0075] Labeling the stream images may further comprise: extracting, from the set of stream images, a set of stream item images, each stream item image depicting one identified item such that the number of stream item images in the set of stream item images may be identical to a number of items identified in the set of stream images; and wherein labeling may be performed on the set of stream item images.

[0076] The set of stream images or the set of stream item images may be used in a variety of ways.

[0077] The method may further comprise: comparing the set of stream images with the set of fraction images, or the set of stream item images with the set of fraction item images, and removing, from the set of stream images or stream item images, images depicting an item which is depicted in any one of the fraction images or fraction item images, thereby producing a modified set of stream images or a modified set of stream item images; labeling each stream image of the modified set of stream images, or each stream item image of the modified set of stream item images, with a label indicative of the depicted item having been sorted into the at least one unsorted fraction; and wherein constructing the at least one classifier may further comprise training the machine learning model using the stream images of the modified set of stream images or the stream item images of the modified set of stream item images captured at the first sorting arrangement and each associated label.

[0078] Hereby, the machine learning model is trained on a larger data set, and not only trained to distinguish items to be sorted into a sorted fraction, but also to distinguish items that is not to be sorted into such a sorted fraction. Thus, a more precis classifier is provided.

[0079] Labeling the fraction images may be performed in a variety of ways.

[0080] Preferably, labeling the fraction images captured at the first sorting arrangement with fraction labels is performed by a processing unit, and wherein the processing unit is further configured to provide the machine learning model with the fraction images captured at the first sorting arrangement and the fraction labels.

[0081] It should be noted that the above may be applicable to the labeling of any other image set, such as the set of stream images or modified stream images. That is, labeling stream images or modified stream images may be performed by a processing unit, preferably the same processing unit that is labeling the fraction images.

[0082] The processing unit may in the context of this disclosure be a central processing unit. Alternatively, the processing unit may comprise a plurality of components arranged separate from and in communicative connection with each other. The processing unit may further be configured to execute the functions of: collecting and compiling the set of fraction images and / or stream images, comparing the set of stream images with the set of fraction images and removing, from the set of stream images, images depicting an item which is depicted in any one of the fraction images, thereby producing a modified set of stream images. Further, the processing unit may be configured to provide the machine learning model with the stream images of the modified set of stream images captured at the first sorting arrangement and each associated label. Furthermore, the processing unit may be configured to execute the function of rejecting outlier images, i.e. images which deviate more than a predetermined level of tolerance from the rest of the set. Rejection of outlier images may optionally be supplemented by manual involvement of an operator.

[0083] The classifier may be distributed to the second sorting arrangement in a variety of ways. For example, the classifier may at the first sorting arrangement be stored on a non-transitory storage medium, which medium is subsequently transported to the second sorting arrangement.

[0084] Preferably, the method further comprises distributing the at least one classifier from the first sorting arrangement to the second sorting arrangement via a network infrastructure, such as a cloud-based server.

[0085] Hereby, the distribution of the at least one classifier may be automated. At the second sorting arrangement, the classifier may, e.g., be implemented on a controller unit. Such controller unit may therefore be arranged in communicative connection with the first sorting arrangement via the same network infrastructure, for example by being in communicative connection with the same cloud-based server.

[0086] The method may further comprise updating the at least one classifier distributed to the second sorting arrangement, wherein updating the at least one classifier distributed to the second sorting arrangement comprises: at the first sorting arrangement: repeating the steps of: sorting a stream of items into at least one sorted fraction and optionally at least one unsorted fraction, capturing a set of fraction images of the items in the at least one sorted fraction, labeling the fraction images captured at the first sorting arrangement with fraction labels, and constructing at least one classifier comprising training the machine learning model using the fraction images captured at the first sorting arrangement and the fraction labels, thereby providing at least one updated classifier; distributing the at least one updated classifier to the second sorting arrangement; and activating the at least one updated classifier at the second sorting arrangement.

[0087] Hereby, at first, the classifier is updated at the first sorting arrangement by subjecting the machine learning model to further learning, which updated classifier is then distributed to further sorting arrangement. Since the classifier need only be updated centrally, i.e. at the first sorting arrangement, a more efficient method is provided.

[0088] That the at least one updated classifier is activated at the second sorting arrangement may, e.g., entail that the at least one classifier of an earlier version is deactivated, i.e. that the updated classifier merely replaces the earlier classifier on a functional level, i.e. as the classifier controlling the sorting process. Hereby, the earlier classifier may still be kept in the system, and may be selectively activated should it be needed. Thus, a more flexible solution is provided.

[0089] Updating the at least one classifier distributed to the second sorting arrangement may further comprise sending a validation request from the second sorting arrangement to the first sorting arrangement, wherein if the validation request is failed, the following steps are performed: at the first sorting arrangement: repeating the steps of: sorting a stream of items into at least one sorted fraction and optionally at least one unsorted fraction, capturing a set of fraction images of the items in the at least one sorted fraction, labeling the fraction images captured at the first sorting arrangement with fraction labels, and constructing at least one classifier comprising training the machine learning model using the fraction images captured at the first sorting arrangement and the fraction labels, thereby providing at least one updated classifier; distributing the at least one updated classifier to the second sorting arrangement; and activating the at least one updated classifier at the second sorting arrangement.

[0090] The validation request may comprise requesting comparison of the machine model version at the first sorting arrangement with the machine model version at the second sorting arrangement. Hence, failing a validation request may be performed if it is determined that the versions are not the same.

[0091] Alternatively, a validation request may be sent from the first sorting arrangement to any further sorting arrangement. That is, an update from the first sorting arrangement may be pushed onto further sorting arrangements.

[0092] Training the machine learning model may further comprise: embedding data of the fraction images captured at the first sorting arrangement and the associated fraction labels and optionally data of the stream images captured at the first sorting arrangement and the associated stream labels as data points in a spatial map, such as a 2-dimensional or a 3-dimensional map; and dividing the spatial map into cells, each cell having a data point density and a date of last entry indicating when the latest data point was added to the cell.

[0093] A data point density of a cell may also be referred to as an embedding density. Embedding data of set of images may be useful for a variety of reasons. For example, repeating the step of constructing at least one classifier may further comprise: for each cell, reviewing the data point density and / or the date of last entry, and if the data density of the cell is equal to or higher than a predetermined threshold, and / or if the date of last entry of the cell is equal to or older than a predetermined threshold, modifying a set of data points, such as removing every data point from the set of data points, contained within the cell.

[0094] That is, if a cell has too high of a data point density, or if the date of last entry is too old, the set of data points contained within the cell may be modified accordingly. For example, if the date of last entry is too old, i.e. too much time has lapsed since a data point was added to the cell, all data points in the cell may be removed. This may for example be beneficial if the data points in the cell represent items which are no longer of interest to sort.

[0095] Additionally or alternatively, repeating the step of constructing at least one classifier may further comprises: when adding a new data point to the spatial map, checking the data point density and / or the date of last entry of the cell into which the data point is to be added, and: if the data density of the cell is equal to or higher than a predetermined threshold, discarding the new data point; and / or if the date of last entry of the cell is equal to or older than a predetermined threshold, adding the new data point and removing at least one neighboring data point in the cell.

[0096] Hereby, when the training dataset is enlarged, data points that are redundant or too old may be discarded. Thus, the at least one classifier may be updated to be able to control sorting of new items, or to control sorting of items in a new way.

[0097] A predetermined threshold value for date of last entry may for example be a week, or month, or a plurality of months, or a year, or a plurality of years, etc.

[0098] A predetermined threshold value for data density may for example be a number a respective number assigned the cell, such as 10, 20 or 100, indicating that the cell may only contain 10, 20 or 100 datapoints. All cells may have the same threshold value. A cell may have a different threshold value as other cells.

[0099] Alternatively, the predetermined threshold value for a cell may be set in accordance with a designated ratio, wherein the ratio is calculated by dividing the total number of data points in all cells with the number of cells. Thus, a predetermined threshold for a cell may be 10 datapoints per cell, 20 datapoints per cell, or 100 datapoints per cell. The exact value may therefore be dependent on how many data points in total the dataset comprised at the time at which the predetermined threshold value was set. Further, the predetermined threshold value may change over time. The predetermined threshold value may e.g. be refreshed every time the at least one classifier is updated, such that each cell is provided with an updated ratio.

[0100] Alternatively, the predetermined threshold may be set in accordance with a probability distribution of data points over the spatial map. That is, a cell to which datapoints are frequently added, or which already comprises a lot of data points, may have a higher predetermined threshold value than a cell to which datapoints are less frequently added, or which comprises less data points. Hereby, when a cell which is assigned a low probability density value is provided with a new data point, every other cell, or at least every other cell with a higher probability density value, may have their predetermined threshold value increased.

[0101] Additionally or alternatively, a dataset may cleaned out before any further data points are added, in particular when updating the at least one classifier. For example, a dataset may be cleaned out by at a specific ratio per cell. For example, 5%, or 10%, or 20%, or 30% of all datapoints for each cells may be removed before any cell may be provided with a further datapoint.

[0102] Any number of classifiers may be constructed. For example, the method according to any one of the preceding claims, wherein constructing at least one classifier comprises constructing a respective classifier for each sorted fraction of the at least one sorted fraction. Moreover, only one classifier may be constructed, wherein the classifier, at the second sorting arrangement, is configured to sort a stream of items into one or more sorted fractions. That is, one classifier is configured to control sorting a stream of items into multiple sorted fractions. Alternatively, more than one classifier may be constructed. For example, one classifier per sorted fraction may be constructed, wherein at the second sorting arrangement, sorting of a stream of items into a plurality of sorted fractions is controlled by a plurality of classifiers. As an example, at the first sorting arrangement, two classifiers may be constructed, such that at the second sorting arrangement, two classifiers are controlling sorting of a stream of items into a respective sorted fraction.

[0103] Any number of classifiers may be distributed to a further sorting arrangement. That is, the number of classifiers constructed at the first sorting arrangement does not have to equal the number of classifiers distributed to a particular sorting arrangement. For example, two classifiers may be constructed at the first sorting arrangement, whereas only one classifier is distributed to the second sorting arrangement.

[0104] Moreover, a classifier may be distributed to more than one further sorting arrangement. Further, the number of classifiers distributed to a second sorting arrangement may differ from the number of classifiers distributed to a third sorting arrangement.

[0105] That is, the method may further comprise: at a third sorting arrangement: capturing images of items in a stream of items; controlling sorting of items in the stream of items into at least one sorted fraction and optionally at least one unsorted fraction using the images captured at the third sorting arrangement and one classifier.

[0106] At any further sorting arrangement, i.e. , for example a second sorting arrangement or a third sorting arrangement, the sorting process may be configured in view of the classifier to be distributed to such sorting arrangement. That is, further sorting arrangement may be constructed ab initio based on the classifier.

[0107] According to one embodiment, the method comprises: sharing the updated database with a second classifier associated with a second arrangement. For instance, a first database associated with a first classifier of a first feeding arrangement may be updated to remove at least one item representation and / or a label corresponding to a type of bottle no longer being relevant to be processed. The first database may subsequently be shared with a second classifier associated with a second arrangement. Said sharing may be performed via a wired connection and / or a wireless connection. Said sharing or distributing may be performed via cloud communication.

[0108] According to one embodiment, the method comprises: updating the at least one classifier based at least on said dataset after said dataset has been updated. Here, if the classifier is a machine learning model, updating the at least one classifier may comprise updating the machine learning model of the at least one classifier. This may advantageously make the at least one classifier more effective at classifying those items which are still relevant to be classified.

[0109] According to one embodiment, the method comprises: sharing the updated classifier with a second feeding arrangement, optionally via cloud communication.

[0110] Cloud communication can be leveraged to share data, e.g., uploading and downloading to a cloud-server via respective communication links. Thus, for the present invention, it may advantageously facilitate updating datasets and / or classifiers.

[0111] According to one embodiment, said dataset is a first dataset, and said feeding arrangement is a first feeding arrangement associated with a first application, such as waste processing or recycling, wherein the method comprises: updating a second dataset based on an update in the first dataset, which second dataset is used by a classifier implemented in a second arrangement associated with a second application, wherein said second application is the same as the first application or different from said first application. The second arrangement may be a second feeding arrangement and / or an arrangement configured to operate on the same kind of items as the first feeding arrangement.

[0112] According to one embodiment, the method comprises: updating the at least one classifier and / or the second dataset based at least on said first dataset after said first dataset has been updated.

[0113] It should also be understood that the first feeding arrangement and / or the second arrangement may be a sorting arrangement. Any one of the first feeding arrangement and the second arrangement may be configured to cooperate with a subsequent sorting arrangement.

[0114] The first feeding arrangement and the second arrangement may be of different designs and / or the same design. The first feeding arrangement and the second arrangement may be configured for processing the same type of items and / or be configured for the same type of application.

[0115] According to one non-limiting example, the first feeding arrangement may be configured for a first application, e.g., inspecting items of a stream of waste material, and the second arrangement may be configured for a second application, e.g., inspecting recyclable items, such as PET bottles. A first classifier associated with the first feeding arrangement may be configured to classify at least one image of a waste item (which may occasionally be a PET bottle) based on a first dataset and a first similarity metric. A second classifier associated with the second arrangement may be configured to classify at least one image of a PET bottle based on a second dataset and a second similarity metric. In other words, according to a non-limiting example, the first feeding arrangement is a waste feeding arrangement. According to a nonlimiting example, the second feeding arrangement is a reverse vending machine, RVM, generally configured to receive and validate used beverage containers, for example PET bottles.

[0116] As may occur, PET bottles may also occur within waste material. Thus, the first dataset may comprise item representations and labels for PET bottles. Then, a manufacturer of a specific PET bottle updates the PET bottle in some way, such as shape and / or material of the PET bottle, thus resulting in that over time, this specific type of PET bottles seizes to appear among the waste material. This fact is then indicated by the aforementioned predetermined time period having lapsed since any item representation for that particular type of PET bottle was last identified. The first database is thereafter updated to remove item representations and / or labels corresponding to this specific PET bottle.

[0117] As it is reasonably that the same type of PET bottle will also seize to appear in the stream of items at the second arrangement, the second dataset may be updated accordingly to remove item representations and / or labels corresponding to the specific type of PET bottle. Optionally, if the second dataset is configured to be updated in the same manner as the first dataset, the second dataset may be updated before the predetermined time period has lapsed for the same item representations and / or labels in the second dataset based on the information received from the first dataset.

[0118] The method may comprise analyzing the first dataset forwarding that information to the second arrangement for updating the second dataset.

[0119] The above example is a non-limiting example only, and the working principle thereof may be applied generally irrespective of applications and items being processed in the respective streams of items or to any one specific selections of first application and second application.

[0120] By implementing the working principle above, generally, or specifically for any one application, the second dataset may be updated faster and / or with fewer intermediate steps to remove irrelevant item representations and / or labels. This may advantageously conform all cooperating datasets so that they are kept up-to-date with one another accordingly, thus streamlining a processing of the same item or items across various applications. Further, it may advantageously speed up the second classifier associated with the second arrangement as it will use an updated second dataset. In particular, the working principle above may result in the second classifier classifying images in an efficient manner without necessarily using the method according to the first aspect to update the second dataset directly.

[0121] According to one embodiment, the method comprises providing feedback data of an inspection of at least one item in the stream of items.

[0122] The feedback data may comprise classification results of the items in the stream of items. Classification results may comprise item representations of items in the stream of items and / or labels (e.g., “plastic,” “metal,” “paper,” or “non-recyclable”) associated with at least one item representation.

[0123] The feedback data may comprise physical properties of the items in the stream of items. Physical properties may comprise dimensions, shape, and / or geometry of items. Physical properties may comprise weight and / or density. Physical properties may comprise color and / or spectral properties. The feedback data may comprise location and / or orientation of items.

[0124] The feedback data may comprise information indicating what action to take for any one item. Such actions may e.g., be to i) sort an item into a first fraction, a second fraction, provide into a non-sorted fraction at anyone sorting arrangement; ii) remove an item from the stream of items by a manual operator; iii) track an item in the stream of items; iv) initiate an safety alert. It should be understood that these are only non-limiting examples and the information may indicate other actions to take.

[0125] The feedback data may comprise error and anomaly data. For instance, information about misclassified items or deviations from expected classifications and / or instances of overlapping, damaged, or occluded items.

[0126] The feedback data may comprise statistical data, such as counts of items in each category over a predetermined time period, and / or such as ratios or percentages of each classification in the stream of items.

[0127] The feedback data may comprise performance metrics, such as efficiency or throughput (e.g., number of items inspected / sorted per minute, hour, day, etc.) and / or such as quality metrics (such as purity of sorted materials or error rates).

[0128] The feedback data may comprise raw sensor readings (e.g., images, spectra, or other sensor outputs provided by the camera arrangement or any other sensors relied upon in the method), and / or processed sensor data, such as item representations.

[0129] The feedback data may comprise ambient factors affecting inspections, such as lighting conditions, temperature, or dust levels.

[0130] The feedback data may comprise historical data, such as trends or patterns over time for specific classifications of items, and / or such as sorting feedback for refining subsequent or precursor sorting processes or classification models (e.g., machine learning models).

[0131] In addition or alternatively, the feedback data may comprise real-time data of blockage in the stream of items in any of at least one feeding arrangement, before said any of at least one feeding arrangement and / or after said any of at least one feeding arrangement. The feedback data may comprise real-time data of blockage in the stream of items in any of at least one sorting arrangement, before said any of at least one sorting arrangement and / or after said any of at least one sorting arrangement. If two or more feeding / sorting arrangements are implemented, the real-time data of blockage may comprise data of blockage in the stream of items between a first feeding / sorting arrangement and a second feeding / sorting arrangement.

[0132] The term “blockage” in the context of the present disclosure refers to a condition where a stream of items is obstructed, disrupted, or completely stopped within a specific part of a feeding / sorting plant comprising said at least one feeding arrangement and / or said at least one sorting arrangement. Blockage can occur due to the accumulation of items, improper alignment of items, mechanical failure, or foreign items getting stuck in conveyors, chutes, feeders, or other machinery. Blockages can result in downtime, reduced efficiency, or damage to the equipment, necessitating immediate identification and resolution to restore normal operations. Blockage can be complete (a stream of items is completely impacted by blockage) or partial (only a part of stream of items is impacted by blockage whereas the remainder of the stream of items can proceed according to expectations).

[0133] Real-time data of blockage may comprise data of at least one blockage occurring at any one of at least one given location. Real-time data of blockage may comprise data indicative of at least one future blockage at one of said at least one given location. The at least one given location may be a monitored location.

[0134] “Monitored location” may refer to a location that is being monitored, continuously or intermittently, by a surveillance camera or a camera arrangement. As an example, a monitored location may be provided by an aforementioned inspection zone.

[0135] According to one aspect of the present invention, at least one feeding arrangement is provided. Any one feeding arrangement of said at least one feeding arrangement may comprise any one feature described in association with the feeding arrangement described above. According to one aspect of the present invention, at least one sorting arrangement is provided. Any one sorting arrangement of said at least one sorting arrangement may comprise any one feature described in association with the first / second / third sorting arrangement described above.

[0136] According to one aspect, a processing plant is provided. The processing plant may comprise said at least one feeding arrangement and / or said at least one sorting arrangement.

[0137] The processing plant may comprise at least one surveillance camera. The first sorting arrangement and / or the second sorting arrangement may comprise at least one camera arrangement. Here, camera arrangement may be an image sensor, such as a VIS camera, or any other type of sensor configured to provide sensor data which can be represented as image data.

[0138] The at least one given location may also be a non-monitored location located somewhere between a first monitored location and a second monitored location. Real-time data of blockage of the non-monitored location may comprise information inferred by a comparison of sensor data of the first monitored location and sensor data of the second monitored location. Here, sensor data may be data provided by the aforementioned camera arrangement or any other inspection arrangement (e.g., a surveillance camera).

[0139] The method may comprise taking blockage and / or risk of blockage into account when updating a configuration of the at least one feeding arrangement and / or the at least one sorting arrangement. The method may comprise updating a feeding configuration of a feeding arrangement to reduce blockage or to reduce a risk of blockage. The feeding configuration may e.g., be related to transport speed through an inspection zone so as to provide sufficient time for camera arrangement to provide qualitatively good sensor data and / or to provide sufficient time for the classifier to classify items in the stream of items. The method may comprise updating a first configuration of a first sorting arrangement to reduce blockage or to reduce a risk of blockage. The method may comprise updating a second configuration of a second sorting arrangement to reduce blockage or to reduce a risk of blockage. When blockage is accounted for, the configurations for the at least one feeding arrangement and / or the at least one sorting arrangement may reduce the risk of blockage and consequently allow for a higher uptime, thus improving the efficiency of the at least one feeding arrangement and / or the at least one sorting arrangement.

[0140] Risk of blockage may be based on at least one parameter, or any combination of a plurality of parameters, that is measured or at least derivable by at least one blockage sensor (e.g., a flow sensor or a proximity sensor) or by at least one virtual blockage sensor (e.g., a VIS-camera arrangement). Here, virtual blockage sensor which is configured to provide sensor data for another purpose other than blockage detection, but said sensor data may still be used to infer blockage.

[0141] The at least one parameter or any one of the plurality of parameters may be any one of item accumulation, time to recovery, throughput impact, sensor feedback tolerance, and system downtime threshold, each of which are described in the following.

[0142] Threshold of item accumulation may indicate or specify a maximum permissible buildup of materials at specific points in the system, quantified in terms of weight, volume, or height, before the situation is classified as a blockage.

[0143] Time to recovery may indicate or specify a time duration for which the at least one feeding arrangement and / or the at least one sorting arrangement can endure an obstruction and resolve it autonomously — using mechanisms such as vibrations, reverse motion, or increased motor power — without requiring manual intervention.

[0144] Throughput impact may indicate or specify a maximum allowable reduction in material throughput (e.g., as a percentage of nominal capacity) that can occur during a partial blockage while maintaining acceptable performance levels.

[0145] Sensor feedback tolerance may indicate or specify an acceptable deviation in sensor readings (e.g., proximity, flow, or pressure sensors) that indicates a blockage without necessitating a system shutdown. System downtime threshold may indicate or specify an acceptable frequency or duration of downtime caused by blockages within a defined operational period (e.g., "no more than 5 minutes of downtime per hour due to blockages").

[0146] A system, which system comprises the at least one feeding arrangement and / or the at least one sorting arrangement, may be configured to determine at least a first blockage tolerance based at least on the first configuration of a first sorting arrangement and / or at least a second blockage tolerance based at least on a second configuration of a second sorting arrangement. Each sorting arrangement may be assigned at least one blockage tolerance. Each blockage tolerance may specify or indicate a maximum acceptable risk of blockage at any given location within the sorting arrangement.

[0147] Blockage tolerance may be a fraction (or percentage) of the system’s maximum allowable risk. The blockage tolerance may be based at least on the maximum risk and a safety margin subtracted from the maximum risk. As a non-limiting example, if the maximum risk is 1.0 (100%) and the safety margin is 0.2 (20%), the blockage tolerance is 0.8 (80%), meaning that the system can tolerate up to 80% of the maximum risk before considering it a blockage. The blockage tolerance, or the system’s maximum allowable risk, may be based at least on a nominal capacity at a given location within the sorting arrangement. The nominal capacity at a given location within the sorting arrangement may depend on the configuration determined for said sorting arrangement. Thus, when updating the configuration for said sorting arrangement, the blockage tolerance may be updated also.

[0148] As mentioned, “Real-time data” may refer to data based on a current value / price / availability of a specific object type, or any other time-dependent variable related to a specific object type. However, “Real-time data” may, alternatively or in combination, refer to visual data of at least one object or a stream of items.

[0149] Visual data may include at least one image and / or at least one video captured by means of said at least one sensor arrangement or said at least one surveillance camera. The at least one sensor arrangement and / or the at least one surveillance camera may be communicatively coupled with the processing unit (e.g., the circuitry in particular) to provide said at least one image and / or said at least one video.

[0150] The system may comprise a monitoring system. The monitoring system may comprise at least one monitor display. The at least one sensor arrangement and / or the at least one surveillance camera may be communicatively coupled with a monitoring system. Each monitor display may be configured to display in at least one window said at least one image or said at least one video. A single monitor display may be configured to display a plurality of images and / or a plurality of videos in respective windows. The monitoring system may be arranged in a monitoring room. An operator stationed in the monitoring room may thus observe the at least monitor display to identify a blockage in said at least one image and / or said at least one video.

[0151] The at least one camera arrangement and / or the at least one surveillance camera may be communicatively coupled with a classifying system. The classifying system may implement a machine learning model trained on labeled training data. The classifying system may comprise a neural network configured to process said at least one image and / or said at least one video as input data and generate an output indicative of whether said at least one image and / or said at least one video depicts a blockage.

[0152] The real-time data of blockage may comprise output from said classification system.

[0153] As mentioned, the classification system may comprise a neural network. The neural network comprises an input layer configured to receive said at least one image and / or said at least one video as input data. The neural network comprises one or more hidden layers, each hidden layer comprising a plurality of nodes configured to process the input data based on a set of weighted connections. The neural network comprises an output layer configured to provide the output based on how the one or more hidden layers processes the input data, which output is indicative of whether said at least one image and / or said at least video depicts a blockage. The neural network may be trained using a training dataset and a learning algorithm to perform a task of detecting whether said at least one image and / or video depicts a blockage. The learning algorithm may be any applicable learning algorithm.

[0154] A classification system according to the invention may comprise any one of, or any combination of: 1 ) at least two fully connected layers; 2) at least one convolutional layer, 3) at least one pooling layer, 4) at least one recurrent layer; 5) at least one attention layer; 6) at least one embedding layer; 7) at least one batch normalization layer; 8) at least one normalization layer and / or at least one group normalization layer; 9) at least one dropout layer; at least one 10) flattened layer; 11) at least one upsampling layer and / or at least one transposed convolution layer, and 12) at least one activation layer. The aforementioned layer(s) are each configured to enhance feature extraction, reduce overfitting, and improve training convergence.

[0155] The training dataset may include at least one labeled image and / or at least one labeled video. The labeled image and / or the labeled video may be provided in various ways, e.g., by image and / or video being labeled in realtime (by an operator) or labeled retroactively. The monitoring system may comprise an interface allowing an operator to provide an input when he / she observes a blockage in at least one image and / or at least one video, thereby allowing the operator to label in real-time said at least one image and / or video as depicting a blockage.

[0156] The monitoring system may be configured to interface with the classification system. The monitoring system may be configured to provide a blockage alert associated with at least one image and / or video if the classification system determines said at least one image and / or video depicts a blockage. This may advantageously alert the operator of a blockage in said one or more flow of materials. In case of a false alert, the operator may override said blockage alert. The monitoring system may be configured to relabel said at least one image and / or video whenever an operator overrides a blockage alert. The monitoring system may be configured to store an anomaly database and update said anomaly databased with entries of anomalies and associated information (date, time, location, event, etc.). The anomaly database may be updated to store entries of blockages. An entry of blockage may comprise information about date, time, and location of a blockage, and also at least a reference to said at least one image and / or video in which a blockage was determined to be depicted. The entry may further comprise information about whether a blockage alert was overridden. The entry may further comprise information about other real-time data.

[0157] The classification system may be configured to train the neural network on a selection of training data, which selection of training data is associated with at least on age and / or a system configuration. Training data associated with a certain age may be omitted from said selection of training data. Training data associated with a certain system configuration may be omitted from said selection of training data. As a non-limiting example, training data associated with a certain image sensor may be omitted if said certain image sensor is replaced.

[0158] “Real-time data” may also comprise auxiliary data of the stream of items at any of the at least one monitored locations. The system may be configured to use auxiliary data in combination with visual data to facilitate training of the classification system or facilitate detection of blockage.

[0159] As a non-limiting example, auxiliary data may comprise material density. High-density items can place a significant load on the system and may cause clogging if the equipment is not designed to handle such materials.

[0160] As a non-limiting example, auxiliary data may comprise object size and shape. Irregularly shaped or oversized items can accumulate in certain areas, leading to obstructions in the stream of items.

[0161] As a non-limiting example, auxiliary data may comprise moisture content. Materials with high moisture contents or sticky properties tend to clump together, which increases the likelihood of blockages in the sorting plant. As a non-limiting example, auxiliary data may comprise throughput rate. Excessive input of materials into the system at a high rate can overwhelm the machinery, creating jams and disrupting operations.

[0162] As a non-limiting example, auxiliary data may comprise conveyor speed. Incorrectly set conveyor speeds can result in material misalignment or excessive buildup, which may eventually block the system.

[0163] As a non-limiting example, auxiliary data may comprise information about component wear and tear. Worn-out or degraded components in the machinery can lead to uneven material handling, which might contribute to clogs and system malfunctions.

[0164] As a non-limiting example, auxiliary data may comprise information indicative of motor load of a conveyor motor. An unexpected or sudden increase in the load on a conveyor motor may indicate that a blockage is present and restricting movement.

[0165] If the system incorporates a pneumatic transport system, which it may, auxiliary data may comprise pressure levels of a pneumatic transport system. In pneumatic transport systems, an increase in pressure levels can signal that airflow is obstructed due to a blockage.

[0166] As a non-limiting example, auxiliary data may comprise information about component temperature. Unusual overheating of specific components in the system can suggest mechanical issues or blockages impeding normal operations.

[0167] As a non-limiting example, auxiliary data may comprise information about vibration patterns. Irregular or abnormal vibration patterns detected in machinery can indicate interruptions in stream of items, often caused by blockages.

[0168] As a non-limiting example, auxiliary data may comprise proximity sensor feedback. Data from proximity sensors may show material accumulation at critical locations, such as choke points, which could suggest the presence of a blockage. As a non-limiting example, auxiliary data may comprise flow sensors data. Flow sensors provide feedback on the movement of materials and can indicate interruptions or abnormalities in the stream of items.

[0169] As a non-limiting example, auxiliary data may comprise information about contaminants or foreign items. The presence of non-sorted items or debris in the stream of items can obstruct the flow, becoming lodged in machinery and causing system downtime.

[0170] A dementia process similar to the dementia process described in the context of the classifier may also be applied for the monitoring system. The classifying system may be configured to implement a dementia process to optimize a dataset used by the classifying system. The dataset may comprise data associated with alerts (such as blockage alerts). Over time, however, some of these alerts occur less frequently or become irrelevant due to updated configurations of the at least one feeding arrangement and / or the at least one sorting arrangement. Thus, the dementia process may be used to forget such alerts to speed up the classifier and optimize its dataset.

[0171] Further, the monitoring system may be configured to send at least one instruction to a robot. The robot may be configured to resolve a blockage based on said at least one instruction. The robot may be a humanoid robot. A humanoid robot may be advantageous since processing plants (such as sorting plants) are generally designed to allow human personnel to navigate the processing plant to perform maintenance and access various feeding locations along any one stream of items to resolve a blockage if necessary. By providing humanoid robots instead, safety is increased. However, said robot being a humanoid robot is a non-limiting example, and it should be understood that other types of robots may be implemented, such as flying drones, or stationary robot arms arranged at various feeding locations.

[0172] Blockage and / or abnormality detection may be supported by any one of the following: Time series prediction based on Al (e.g., DARTS); RNN- based Models (LSTM, GRU); Temporal Convolutional Network (TCN), and / or Transformer Models for time series (N-BEATS). Time series prediction involves forecasting future values of data that are recorded sequentially over time. With advancements in artificial intelligence (Al), time series forecasting may employ sophisticated machine learning (ML) and deep learning (DL) algorithms rather than traditional statistical models. An Al-based model may be particularly adept at handling complex datasets, capturing temporal dependencies, and identifying patterns that may not be apparent through conventional methods. Thus, time series prediction based on Al may allow prediction of blockages and / or abnormality detections. As a non-limiting example, blockage and / or abnormality detection may be implemented based on Demand Forecasting and Time Series (DARTS), which is an open-source Python library for time series prediction and forecasting using machine learning and deep learning.

[0173] RNN-based models, such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), may be implemented for blockage and / or abnormality detections, as they are particularly effective at capturing temporal dependencies and patterns in data.

[0174] TCNs are convolutional neural networks (CNNs) designed specifically for sequential data. Unlike RNNs, which process sequences step by step, TCNs use convolutional layers to capture patterns across multiple time steps in parallel.

[0175] Transformer models may be used for blockage and / or abnormality detection due to their ability to capture complex relationships in temporal data sequences using an attention mechanism. Instead of processing data sequentially like RNNs, transformers models may consider all time steps simultaneously, making them highly parallelizable and scalable.

[0176] According to a second aspect of the present inventive concept, a computer readable storage medium is provided, the medium having stored thereon instructions for implementing the method according to the first aspect of the present invention.

[0177] Any benefit or technical effect discussed in relation to the first aspect of the present inventive concept may be applicable to the second aspect of the present inventive concept, and will therefore not be mentioned in order to avoid undue repetition.

[0178] Moreover, while the above method is described in relation to a feeding arrangement, the method may be performed independently of the feeding arrangement on the dataset as such. In other words, the method may involve reviewing the dataset in terms of timestamps and remove any item representations and / or label having a timestamp indicating that the predetermined time period has lapsed.

[0179] Brief Description of the Drawings

[0180] The inventive concept will in the following be described in more detail with reference to the enclosed drawings, wherein:

[0181] Fig. 1 is a schematic view of a system implementing the method according to the first aspect of the present inventive concept, wherein a first sorting arrangement is shown, the first sorting arrangement being communicatively connected to a processing unit, and a second sorting arrangement comprising a controller being communicatively connected to the processing unit of the first sorting arrangement;

[0182] Fig. 2 is a flow chart illustrating a portion of the method according to one embodiment of the present inventive concept;

[0183] Fig. 3A-3B are schematic illustrations of a spatial embedding of data;

[0184] Fig. 4 is a perspective view of an exemplifying second sorting arrangement for sorting textiles;

[0185] Fig. 5 is a flow chart illustrating the method according to the first aspect of the present inventive concept

[0186] Fig. 6 is a schematic view of a system implementing the method according to the first aspect of the present inventive concept, wherein a feeding arrangement is shown, the feeding arrangement being communicatively connected to a processing unit, and

[0187] Fig. 7 is a schematic illustration illustrating a dementia process of the present inventive concept. Detailed Description of the Drawings

[0188] The present inventive concept will be described more fully hereinafter with reference to the accompanying drawings, in which preferred variants of the inventive concept are shown. This inventive concept may, however, be varied in many different ways and should not be construed as limited to the variants set forth herein; rather, these variants are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept to those skilled in the art. In the drawings, like numbers refer to like elements.

[0189] In Fig. 1 , a sorting process at a first sorting arrangement 1 is schematically illustrated. Here, an input stream of items 2 is provided to the first sorting arrangement 1 .

[0190] The sorting arrangement 1 may implement manual sorting or automatic sorting, or any combination therebetween. For example, the sorting arrangement 1 may comprise a conveyor belt transporting the input stream of items 2, at which conveyor belt operators carrying out a manual sorting process are situated. Alternatively, the sorting arrangement 1 may comprise a conveyor belt transporting the input stream of items 2 being monitored by an automated sorting setup. An exemplary sorting arrangement is shown in and discussed in relation to Fig. 4.

[0191] The stream of items 2 may comprise any type of items or any combination of items, such as clothing, minerals, foodstuff, empty containers, such as metal cans, etc. That is, the items in the stream of items 2 may be made of or comprise plastics, metal, textile material, organic material, mineral material, etc.

[0192] At the first sorting arrangement 1 , the stream of items 2 is sorted into at least one sorted fraction 3. Here, the stream of items 2 is sorted into a first and a second sorted fraction 3. However, the steam of items 2 may be sorted into any number of sorted fractions, i.e. into sorted fractions 1 through N, where N may be any number. As is illustrated in Fig. 1 , the stream of items 2 is also sorted into an optional unsorted fraction 4. The unsorted fraction 4 may constitute items which are to be discarded from the sorting process. Additionally or alternatively, the unsorted fraction 4 may be recirculated into the sorting arrangement 1 , or be provided to any other sorting arrangement.

[0193] In an exemplary sorting arrangement 1 , the stream of items 2 is provided in the form of a stream of clothing comprising pants of different colors, wherein pants of a first color are sorted into the first sorted fraction 3, whereas pants of a second color are sorted into the second sorted fraction 3. Optionally, clothing of any other color than the first and second color will be sorted into the unsorted fraction 4.

[0194] In another exemplary sorting arrangement 1 , the stream of items 2 is provided in the form of a stream of clothing of different types, wherein pants are sorted into the first sorted fraction 3, whereas shirts are sorted into the second sorted fraction 3. Optionally, clothing of any other type than pants and shirts may be sorted into the unsorted fraction 4.

[0195] In yet another exemplary sorting arrangement 1 , the stream of items 2 is provided in the form of objects made of different materials, wherein plastic objects are sorted into the first sorted fraction 3, whereas metal objects are sorted into the second sorted fraction 3. Optionally, objects made of any other material may be sorted into the unsorted fraction 4.

[0196] As is further illustrated in Fig. 1 , a camera arrangement 5 is observing the two sorted fractions 3. The camera arrangement 5 may comprise any camera known in the art. The camera arrangement 5 is configured to capture a set of fraction images of the items in the first and second sorted fractions 3. Capturing the fraction images may be performed at any point after the items have been separated from the input stream of items 2. For example, if the sorting arrangement 1 comprises a conveyor belt for providing the stream of items 2 to the location for physical sorting, wherein after such physical sorting step, each sorted fraction is transported on a separate conveyor belt, a set of fraction images may be captured at any instance at which the items of the sorted fraction are present on such separate conveyor belt. As another example, a set of fraction images may be captured when the items in a sorted fraction 3 have been separated from the stream of items, but are stationary. It is to be understood that the set of fraction images may comprise any non-zero number of images. For example, a set of fraction images may comprise one or more images, such as 10 images, or 100 images, or 1000, or 10000 images. More than one fraction image may be taken of any one item. Consequently, an item may be present in more than one image. Further, more than one item may be present in any one figure.

[0197] The method may further comprise: extracting, from the set of fraction images a set of fraction item images. Each fraction item image may depict one identified item such that the number of fraction item images in the set of fraction item images may be identical to a number of items identified in the set of fraction images. For example, from an image depicting three items may be extracted three item images. Labeling may be performed on the set of fraction item images, and wherein training the machine learning model may be performed using the fraction item images captured at the first sorting arrangement and the fraction labels. Hereby, if an image depicts more than one items, a set of item images may be extracted from the image, thereby producing a cleaner training dataset. Further, a fraction item image may have a smaller data size than the original fraction image. Hereby, a set of images which is easier to process is provided.

[0198] The set of fraction images or the set of fraction item images is provided to a processing unit 6. The processing unit 6 may be a central processing unit. Alternatively, the processing unit 6 may comprise a plurality of components arranged separate from and in communicative connection with each other. The processing unit 6 is arranged in communicative contact with the camera arrangement 5 observing the two sorted fractions 3. Hence, the processing unit 6 may further be configured for controlling the camera arrangement 5, and / or retrieve data, such as the set of fraction images, from the camera arrangement 5.

[0199] Moreover, here, the processing unit 6 is configured to label the fraction images captured at the first sorting arrangement 1 with fraction labels. A fraction label is indicative of the sorted fraction 3 into which a depicted item has been sorted. That is, a label may, e.g., be a keyword, such as “green”, or “pants”, or “aluminum.”

[0200] The processing unit 6 has implemented thereon a machine learning model. The machine learning model may be any type of machine learning model. For example, the machine learning model may be a neural network, such as a convolution neural network or a fully connected neural network. The processing unit 6 is further configured to provide the machine learning model with the fraction images captured at the first sorting arrangement 1 and the fraction labels. Hereby, the processing unit 6 is configured to construct at least one classifier configured to correlate items with fractions. Constructing the at least one classifier comprises training the machine learning model using the fraction images captured at the first sorting arrangement 1 and the fraction labels.

[0201] A further camera arrangement 7 is observing the input stream of items 2. The further camera arrangement 7 is configured to capture a set of stream images of the items in the stream of items 2 before sorting the stream of items 2 into the two sorted fractions 3 and the optional unsorted fraction 4. It is to be understood that the set of stream images may comprise any non-zero number of images. For example, the set of stream images may comprise one or more images, such as 10 images, or 100 images, or 1000 images.

[0202] Here, the processing unit 6 is further be configured to compare the set of stream images with the set of fraction images, and remove, from the set of stream images, images depicting an item which is depicted in any one of the fraction images, thereby producing a modified set of stream images. Further, the processing unit 6 is configured to label each stream image of the modified set of stream images with a label indicative of the depicted item having been sorted into the unsorted fraction 4. Furthermore, constructing the at least one classifier further comprises training the machine learning model using the stream images of the modified set of stream images captured at the first sorting arrangement and each associated label. That is, the processing unit 6 is further configured to provide the modified set of stream images to the machine learning model, in order to construct the classifier. It should be noted that the above described steps of comparing and removing, thereby producing a modified set of images, and the subsequent step of labeling images of such modified set of images, may be applicable the sets of item images as well, i.e. sets of fraction item images and stream item images.

[0203] The processing unit 6 may be further configured to, for each image, identifying at least one depicted item, wherein the label associated with the image is further indicative of the identified item. Identifying a depicted item may, e.g., comprise determining a class or type of the item. For example, identifying a depicted item may comprise determining that the item is of the type “pants”, or “shirts”, or “red pants”, or “blue shirt”, or “can”, or “aluminum can”, etc. Further, it is to be understood that a fraction image may depict more than one item. Thus, labeling the fraction images may comprise identifying, for any one image, more than one item. Identifying a depicted item may be performed in a variety of ways. For example, identifying the depicted item may be performed by foreground-background segmentation. Alternatively, Identifying the depicted item may be performed by object instance segmentation. Object instance segmentation may be performed by a machine learning model. For example, the machine learning model may be part of or be the same machine learning model as the classifier. As such, object instance segmentation may be performed by a neural network, such as a convolution neural network or a fully connected neural network. After an image has been processed by a machine learning model using object instance segmentation, it may be provided to the classifier as training material. Images of any set of images may be subject to further image processing steps. For example, images of the set of fraction images may be subject to image improvement, cropping, etc. Furthermore, it is to be understood that a captured image may be discarded from the respective set, for example if no object can be identified, or if the image is of too low quality.

[0204] Furthermore, the processing unit 6 may be configured to execute the function of rejecting outlier images, i.e. images which deviate more than a predetermined level of tolerance from the rest of the set. Rejection of outlier images may optionally be supplemented by manual involvement of an operator.

[0205] On the lower half of Fig. 1 , a second sorting arrangement 10 is shown, the second sorting arrangement 10 sorting an input stream of items 12. The second sorting arrangement 10 is spatially separated from the first sorting arrangement 1 . This is to be understood as the first and second sorting arrangements 1 , 10 sorting different streams of items 2, 12. The first sorting arrangement 1 may be arranged at a first sorting plant, whereas the second sorting arrangement 10 may be arranged at a second sorting plant. Alternatively, the first and a second sorting arrangements 1 , 10 may be arranged at the same sorting plant. The method may thus be implemented for sorting arrangements located at different parts of a plant, different parts of a city, different parts of a country, in different countries, and / or on different continents.

[0206] The second sorting arrangement 10 is communicatively connected to a controller 16. The controller is communicatively connected to the processing unit 6 of the first sorting arrangement 1 . The controller 16 may be connected to the processing unit 6 in a variety of ways, for example, via a network infrastructure, for example by being in communicative connection via a cloudbased server. The classifier may therefore be distributed from the processing unit 6 to the controller 16.

[0207] The controller 16 may thus implement the machine learning model of the classifier in order to control the sorting process of the second sorting arrangement 10. Here, the input stream of items 12 is sorted into a first and a second sorted fraction 13, and optionally an unsorted fraction 14. Thus, the input stream of items 12 of the second sorting arrangement 10 is sorted into the same number of sorted fractions as the input stream of items 2 of the first sorting arrangement 1 . Alternatively, the stream of items 2, 12 may be sorted into different numbers of sorted fractions 3, 13. That is, the stream of items 2 at the first sorting arrangement 1 may be sorted into two sorted fractions 3, whereas the stream of items 12 at the second sorting arrangement 10 may be sorted into one sorted fraction 13. In Fig. 2, a flow chart of a portion of an exemplary method according to one embodiment of the present inventive concept is illustrated. At the first sorting arrangement 1 , sorting S1 a stream of items 2 into at least one sorted fraction 3 and optionally at least one unsorted fraction 4 is performed. Thereafter, capturing S2 a set of fraction images of the items in the at least one sorted fraction 3 is performed, whereafter labeling S3 the fraction images captured at the first sorting arrangement 1 with fraction labels is performed. Each fraction label is indicative of the sorted fraction 3 of the at least one sorted fraction into which a depicted item has been sorted. Labeling S3 the fraction images may further comprises: for each fraction image, identifying at least one depicted item, wherein the fraction label is further indicative of the identified item. Identifying the at least one depicted item may, e.g., be performed by object instance segmentation.

[0208] Thereafter, constructing S4 at least one classifier configured to correlate items with fractions is performed. The at least one classifier is a machine learning model, and constructing S4 the at least one classifier comprises training the machine learning model using the fraction images captured at the first sorting arrangement 1 and the fraction labels. Constructing S4 at least one classifier may comprises constructing a respective classifier for each sorted fraction 3 of the at least one sorted fraction 3.

[0209] As is illustrated in Fig. 2, the method may optionally include: at the first sorting arrangement 1 , capturing S31 a set of stream images of the items in the stream of items 2 before sorting the stream of items 2 into at least one sorted fraction 3 and optionally at least one unsorted fraction 4. Further, after capturing S31 a set of stream images, comparing S32 the set of stream images with the set of fraction images may be performed. Thereafter, removing S33, from the set of stream images, images depicting an item which is depicted in any one of the fraction images may be performed, thereby producing a modified set of stream images. Thereafter, labeling S34 each stream image of the modified set of stream images with a label indicative of the depicted item having been sorted into the at least one unsorted fraction 4 is performed. Hereby, constructing S5 the at least one classifier may further comprise training the machine learning model using the stream images of the modified set of stream images captured at the first sorting arrangement 1 and each associated label.

[0210] The method further comprises, at a second sorting arrangement 10: capturing S5 images of items in a stream of items 12, and controlling S6 sorting of items in the stream of items 12 into at least one sorted fraction 13 and optionally at least one unsorted fraction 14 using the images captured at the second sorting arrangement 10 and the at least one classifier.

[0211] The method may further comprise distributing S7 the at least one classifier from the first sorting arrangement 1 to the second sorting arrangement 10 via a network infrastructure, such as a cloud-based server.

[0212] The method may further comprise updating S8 the at least one classifier distributed to the second sorting arrangement 10. Updating S8 the at least one classifier distributed to the second sorting arrangement comprises: at the first sorting arrangement 1 : repeating the steps of: sorting S1 , capturing S2, labeling S3 and constructing S4, thereby providing at least one updated classifier. Furthermore, updating S8 comprises distributing the at least one updated classifier to the second sorting arrangement 10, and activating the at least one updated classifier at the second sorting arrangement 10.

[0213] Updating S8 the at least one classifier distributed to the second sorting arrangement 10 may further comprise sending a validation request from the second sorting arrangement 10 to the first sorting arrangement 1 , wherein if the validation request is failed, the steps of sorting S1 , capturing S2, labeling S3 and constructing S4 are performed.

[0214] Updating S8 the at least one classifier may also further comprise training the machine learning model by letting the machine learning model selectively forget data. This may be implemented in a variety of ways. For example, training the machine learning model may further comprise: embedding data of the fraction images captured at the first sorting arrangement and the associated fraction labels and optionally data of the stream images captured at the first sorting arrangement and the associated stream labels as data points in a spatial map, such as a 2-dimensional or a 3- dimensional map; and dividing the spatial map into cells, each cell having a data point density and a date of last entry indicating when the latest data point was added to the cell. A data point density of a cell may also be referred to as an embedding density. An exemplary 2-dimensional spatial embedding map 20 is illustrated in Figs. 3A-B.

[0215] Embedding data of set of images may be useful for a variety of reasons. For example, repeating the step of constructing at least one classifier may further comprise: for each cell, reviewing the data point density and / or the date of last entry, and if the data density of the cell is equal to or higher than a predetermined threshold, and / or if the date of last entry of the cell is equal to or older than a predetermined threshold, modifying a set of data points, such as removing every data point from the set of data points, contained within the cell. Additionally or alternatively, repeating the step of constructing at least one classifier may further comprises: when adding a new data point to the spatial map, checking the data point density and / or the date of last entry of the cell into which the data point is to be added, and: if the data density of the cell is equal to or higher than a predetermined threshold, discarding the new data point; and / or if the date of last entry of the cell is equal to or older than a predetermined threshold, adding the new data point and removing at least one neighboring data point in the cell.

[0216] Hereby, when the training dataset is enlarged, data points that are redundant or too old may be discarded. Thus, the at least one classifier may be updated to be able to control sorting of new items, or to control sorting of items in a new way.

[0217] It is to be understood that the above described method may be implemented for more than one further sorting arrangement. For example, the following steps may be performed: at a third sorting arrangement: capturing images of items in a stream of items; controlling sorting of items in the stream of items into at least one sorted fraction and optionally at least one unsorted fraction using the images captured at the third sorting arrangement and one classifier.

[0218] Instructions for implementing the exemplary method as described above, and any variant thereof, may be stored on a computer readable storage medium. The instructions may thereby be utilized by a processing unit 6 in order to execute the method.

[0219] In Fig. 3A, a spatial embedding of data as data points 21 in a spatial 2- dimensional map 20 is illustrated. The data may, e.g., be derived from the set of fraction images or the set of stream images, or the set of fraction item images or the set of stream item images. That is, a data point 21 may be representative of an item depicted in a fraction image and / or a stream image. That is, a data point 21 may additionally or alternatively be representative of an item depicted in an item image, such as a fraction item image or a stream item image. Hereby, the placement of a data point 21 in the map 20 may be a function of the class to which the item belongs. To this end, the data points may be arranged in clusters 22, where each cluster 22 therefore representing a class of items. In Fig. 3A, data points 21 belonging to the same cluster 22 are indicated by the same gray-scale shade.

[0220] In Fig. 3B, a grid has been applied to the map 20. The grid divides the map 20 into cells 23. Each cell 23 has a data point density. Furthermore, each cell 23 may have an associated date of last entry, indicating when the latest data point was added to the cell 23. Repeating the step of constructing at least one classifier may thus further comprise: for each cell 23, reviewing the data point density and / or the date of last entry, and if the data density of the cell 23 is equal to or higher than a predetermined threshold, and / or if the date of last entry of the cell 23 is equal to or older than a predetermined threshold, modifying a set of data points 21 , such as removing every data point 21 from the set of data points 21 , contained within the cell 23. That is, if a cell 23 has too high of a data point density, or if the date of last entry is too old, the set of data points 21 contained within the cell may be modified accordingly. For example, if the date of last entry is too old, i.e. too much time has lapsed since a data point 21 was added to the cell 23, all data points 21 in the cell 23 may be removed. This may for example be beneficial if the data points in the cell represent items which are no longer of interest to sort.

[0221] Additionally or alternatively, repeating the step of constructing at least one classifier may further comprises: when adding a new data 21 point to the spatial map 20, checking the data point density and / or the date of last entry of the cell 23 into which the data point 21 is to be added, and: if the data density of the cell 23 is equal to or higher than a predetermined threshold, discarding the new data point 21 ; and / or if the date of last entry of the cell 23 is equal to or older than a predetermined threshold, adding the new data point 21 and removing at least one neighboring data point 21 in the cell 23.

[0222] A predetermined threshold value for date of last entry may for example be a week, or month, or a plurality of months, or a year, or a plurality of years, etc.

[0223] A predetermined threshold value for data density may for example be a number a respective number assigned the cell 23, such as 10, 20 or 100, indicating that the cell 23 may only contain 10, 20 or 100 datapoints. All cells 23 may have the same threshold value. A cell 23 may have a different threshold value as other cells 23.

[0224] Alternatively, the predetermined threshold value for a cell 23 may be set in accordance with a designated ratio, wherein the ratio is calculated by dividing the total number of data points 21 in all cells 23 with the number of cells 23. Thus, a predetermined threshold for a cell may be 10 datapoints 21 per cell, 20 datapoints 21 per cell 23, or 100 datapoints 21 per cell 23. The exact value may therefore be dependent on how many data points 21 in total the dataset comprised at the time at which the predetermined threshold value was set. Further, the predetermined threshold value may change over time. The predetermined threshold value may e.g. be refreshed every time the at least one classifier is updated, such that each cell 23 is provided with an updated ratio.

[0225] Alternatively, the predetermined threshold may be set in accordance with a probability distribution of data points 21 over the spatial map 20. That is, a cell 23 to which datapoints 21 are frequently added, or which already comprises a lot of data points 21 , may have a higher predetermined threshold value than a cell 23 to which datapoints 21 are less frequently added, or which comprises less data points. Hereby, when a cell 23 which is assigned a low probability density value is provided with a new data point 21 , every other cell 23, or at least every other cell 23 with a higher probability density value, may have their predetermined threshold value increased.

[0226] Additionally or alternatively, a dataset may cleaned out before any further data points 21 are added, in particular when updating the at least one classifier. For example, a dataset may be cleaned out by at a specific ratio per cell 23. For example, 5%, or 10%, or 20%, or 30% of all datapoints 21 for each cells 23 may be removed before any cell may be provided with a further datapoint 21.

[0227] Fig. 4 illustrates a perspective schematic view of an exemplifying second sorting arrangement 700 having implemented therein the at least one classifier, i.e the at least one classifier has been distributed to this second sorting arrangement 700.

[0228] The sorting arrangement 700 is fed with a stream of items 710, e.g. textile material e.g. clothes. The material 710 is conveyed through a inspection zone 720 by a transporter such as a conveyor belt or slide. However, material may be provided through the inspection zone 720 by any suitable means or manually without any technical means. A light source arrangement 730 and an inspection system 740 comprising a camera system 750 and optionally a spectrometer system 760 is provided in provided inside the housing 222. The camera system 750 and the optional spectrometer system 760 is / are adapted to receive and analyze light, which light is emitted by the light source arrangement 730 and thereafter reflected and / or scattered by the material in the inspection zone 720. The light source arrangement 730 typically emits a spectrum within the UV, VIS and / or NIR wavelength range. The spectrometer system 760 typically acquires a spectrum within the UV, VIS and / or NIR wavelength range; and the camera system 750 typically acquires images within the UV, VIS and / or NIR wavelength range. The inspection system 740 of the sorting arrangement 700 is configured to discriminate one group of textile material from other materials based on the acquired spectrum and / or the acquired images. In other words, the system 740 may be set up such that a specific type of material or groups of materials is discriminated form other types of the materials based on its spectrum, including its color, and / or based on its shape, size or any other detectable appearance.

[0229] The sorting arrangement 700 may optionally comprise a laser triangulation system 746 configured to determine height information related to the material that is conveyed through the inspection zone 720.

[0230] Furthermore, the sorting arrangement 700 preferably comprises an ejection system 224 (such as a robotic arm or nozzles ejecting pressurized air) for sorting the material into different fractions based on the images captured by the camera system and the at least one classifier. That is, the at least one classifier may be implemented on a controller 16 arranged in the sorting arrangement 700, which controller 16 also is configured to execute functions of physically sorting items, such as actuating a robotic arm or a nozzle. Such controller 16 may therefore be communicatively connected to the light source arrangement 730 and the inspection system comprising a camera system 750 and optionally the spectrometer system 760, and additionally may be configured to actuate such systems. Additionally or alternatively, the information received from the inspection system and the optional triangulation system may also be used to verify the sorting decision made based on the classifier, or to enabling a sorting in even finer fractions. Hence, the images captured at the second sorting arrangement 700 and the at least one classifier is used when sorting a stream of items into at least one sorted fraction. A more detailed description of a sorting arrangement 700, including a light source arrangement, an inspection system, a laser triangulation systema and an ejection system 224 may be found in WO 2015 / 063300, which is hereby incorporated by reference.

[0231] In Fig. 5, a flow chart of an exemplary method according to the first aspect of the present inventive concept is illustrated. The method comprises: at a feeding arrangement 1 : feeding ST a stream of items 2 through an inspection zone; capturing S2’ a set of images of the items when in the inspection zone; providing S3’ a dataset comprising item representations and labels; providing S4’ at least one classifier configured to classify at least one image of said set of images based on a similarity metric between said at least one image and at least one item representation of said dataset; classifying S5’ at least one image using at least one classifier of said at least one classifier and the dataset; associating S6’ each item representation and / or label in said dataset with a respective timestamp, and updating S7’ the respective timestamps whenever an associated item representation is identified as a match by at least one of said at least one classifier, and updating S8’ the dataset by removing any item representations and / or labels having an associated timestamp indicating a predetermined time period has lapsed since said item representation was last identified by the classifier to be a match.

[0232] As a non-limiting example, a feeding arrangement 1 of the method described above may comprise an inspection zone and a transporter, e.g., as discussed in relation to Fig. 4. The feeding arrangement may comprise a light source arrangement 730 and an inspection system 740, e.g., as discussed in relation to Fig. 4.

[0233] The feed arrangement is illustrated in Fig. 6, wherein it provides a stream of stream of items, e.g. textile material e.g., clothes. The material is transported as a stream of items 2. A camera arrangement 7 is observing an inspection zone through which the stream of items 2 is provided. The camera arrangement is configured to capture a set of inspection images of the items in the stream of items when in the inspection zone. It is to be understood that the set of inspection images may comprise any non-zero number of images. For example, the set of inspection images may comprise one or more images, such as 10 images, or 100 images, or 1000 images.

[0234] The camera arrangement 7 is communicatively connected to a processing unit 6. The processing unit 6 is configured to execute an implementation of the at least one classifier to classify at least one inspection image provided by the camera arrangement 7 based on a similarity metric between said at least one inspection image and at least one item representation and label of a provided dataset. The at least one classifier may determine said similarity metric between an item representation of said inspection image and said at least one item representation of said dataset.

[0235] The dataset may be stored on a computer readable storage medium, either locally or at a remote server (e.g., a cloud server) the at least one classifier may access.

[0236] Thus, as the method is performed, each item representation and / or label of the dataset is updated with a respective timestamp whenever an associated item representation is identified as a match by at least one of said at least one classifier. The processing unit may be configured to update the dataset by removing any item representations and / or labels having an associated timestamp indicating a predetermined time period has lapsed since said item representation was last identified by the classifier to be a match.

[0237] The predetermined time period may be any suitable time period. As non-limiting examples, the predetermined time period may be any number of hours, days, weeks, months, etc. Generally, the predetermined time period may be such that once lapsed for any item representation and / or label, it is probable that the stream of items does not comprise anymore items which the classifier will identify to match with the item representation and / or labels that will be removed via the aforementioned dementia process.

[0238] In a further embodiment, said feeding arrangement comprises at least one sorting arrangement 1 , 10 configured to sort said stream of items into at least one sorted fraction 3 and optionally at least one unsorted fraction 4, the method comprising: at the at least one sorting arrangement 1 , 10: controlling sorting of items in the stream of items 12 into said at least one sorted fraction 13 and optionally said at least one unsorted fraction 14 using said set of images and said at least one classifier.

[0239] The similarity metric is any one of: a distance-based metric, a histogram-based metric, a statistical metric, a structural or visual similarity metric, and a neural network-learned metric. As a non-limiting example, a histogram-based metric may be implemented.

[0240] The method may further comprise distributing the at least one classifier from the first feeding arrangement 1 to a second feeding arrangement via a network infrastructure, such as a cloud-based server.

[0241] Fig. 7 is a schematic illustration illustrating a non-limiting example dementia process of the present inventive concept. Two instances of a dataset 100 used by the at least one classifier is shown. To the left in Fig. 7, a dataset 100 provided on a computer-readable storage medium comprises a first item representation 101 , a second item representation 102, a third item representation 103, a fourth item representation 104, and a fifth item representation 105. Additionally, the computer-readable storage medium has further unused storage capacity for storing further item representations. As exemplified in Fig. 7, the first item representation 101 corresponds to a first type of clothing and the fifth item representation 105 corresponds to a second type of clothing. As the classifier classifies item representations based on inspection images based on said similarity metric and said dataset 100, a timestamp is associated with each item representation and / or label.

[0242] After a predetermined time period has lapsed, during which time period, the classifier identifies no items in the inspection images to be items corresponding to the fifth item representation 105. Since the predetermined time period has lapsed, which indicates that it is probable that items corresponding to the fifth item representation 105 no longer occurs in the stream of images, and / or e.g., that the fifth item representation 105 is redundant in view of the other four item representations and / or e.g., the fifth item representation 105 is no longer of interest to keep track of, the dataset 100 is updated via the aforementioned dementia process to remove the fifth item representation 105 (as seen to the right in Fig. 7).

[0243] In the above, a method implementing a dementia process is described. While the method is exemplified to strictly remove irrelevant or redundant item representations from the dataset, the method may implement steps of adding new item representations to the dataset, e.g., based on said fraction images, manual labeling, item representations based on new training data etc.

[0244] In the drawings and specification, there have been disclosed preferred variants and examples of the inventive concept and, although specific terms are employed, they are used in a generic and descriptive sense only and not for the purpose of limitation, the scope of the inventive concept being set forth in the following claims.

Claims

CLAIMS1 . A method of inspecting items in a feeding arrangement, the method comprising: at a feeding arrangement (1 ): feeding (ST) a stream of items through an inspection zone; capturing (S2’) a set of images of the items when in the inspection zone; providing (S3’) a dataset (100) comprising item representations (101 , 102, 103, 104, 105) and labels, providing (S4’) at least one classifier configured to classify at least one image of said set of images based on a similarity metric between said at least one image and at least one item representation (101 , 102, 103, 104, 105) of said dataset (10), classifying (S5’) at least one image using at least one classifier of said at least one classifier and the dataset (100); associating (S6’) each item representation (101 , 102, 103, 104, 105) and / or label in said dataset (100) with a respective timestamp, and updating (S7’) the respective timestamps whenever an associated item representation (101 , 102, 103, 104, 105) is identified as a match by at least one of said at least one classifier, and updating (S8’) the dataset by removing any item representations (101 , 102, 103, 104, 105) and / or labels having an associated timestamp indicating a predetermined time period has lapsed since said item representation (101 , 102, 103, 104, 105) was last identified by the classifier to be a match.

2. The method according to claim 1 , wherein said classifying at least one image using the at least one classifier of said at least one classifier and the dataset comprises: identifying among said item representations of said data set at least one candidate item representation based on said similarity metric betweensaid at least one image and at least one item representation (101 , 102, 103, 104, 105) of said dataset (10); determining at least one matching item representation of said at least one candidate item representations based on a predetermined matching condition, and associating said at least one image with one or more labels associated with said at least one matching item representation.

3. The method according to claim 2, wherein said predetermined matching condition comprises at least one of: nearest neighbor approach, nearest plurality of neighbor approach, or histogram threshold approach.

4. The method according to any one of the preceding claims, wherein the feeding arrangement comprises at least one of a chute, a conveyer belt, and a free falling path, wherein said inspection zone is provided at one of said at least one of a chute, a conveyer belt, and free falling path.

5. The method according to any one of the preceding claims, wherein said stream of items comprises any one of, or any combination of: minerals, food, waste, clothing, and recyclables.

6. The method according to any one of the preceding claims, wherein said feeding arrangement comprises at least one sorting arrangement (1 , 10) configured to sort said stream of items into at least one sorted fraction (3) and optionally at least one unsorted fraction (4), the method comprising: at the at least one sorting arrangement (1 , 10): controlling sorting of items in the stream of items (12) into said at least one sorted fraction (13) and optionally said at least one unsorted fraction (14) using said set of images and said at least one classifier.

7. The method according to any one of the preceding claims, wherein said at least one classifier is a machine learning model trained on item representations and labels.

8. The method according to claim 7, wherein said at least one classifier is a machine learning model trained on the item representations and the labels of said dataset.

9. The method according to any one of claims 7 and 8, wherein said item representations are images, or feature representations provided based on images.

10. The method according to any one of claims 1-9, wherein said similarity metric is any one of: a distance-based metric, a histogram-based metric, a statistical metric, a structural or visual similarity metric, and a neural network- learned metric.11 . The method according to any one of the preceding claims, further comprising distributing the at least one classifier from the first feeding arrangement (1) to a second feeding arrangement via a network infrastructure, such as a cloud-based server.

12. The method according to any one of claims 1-11 , wherein the feeding arrangement comprises at least a first sorting arrangement configured to sort said stream of items into at least one sorted fraction (3) and optionally at least one unsorted fraction (4), said inspection zone is provided at said first sorting arrangement, the method comprising: at the first sorting arrangement (1 ): sorting (S1) said stream of items (2) into said at least one sorted fraction (3) and optionally said at least one unsorted fraction (4); wherein said capturing a set of images comprises capturing (S2) a set of fraction images of the items in the at least one sorted fraction (3)when in said inspection zone of said sorting arrangement; labeling (S3) the fraction images captured at the first sorting arrangement (1 ) with fraction labels, wherein each fraction label is indicative of the sorted fraction (3) of the at least one sorted fraction (3) into which a depicted item has been sorted; wherein said providing said dataset comprises providing a dataset comprising at least item representations based on said fraction images and labels based on said fraction labels, wherein said providing at least one classifier comprises constructing (S4) said at least one classifier configured to correlate items with fractions, the at least one classifier being a machine learning model and wherein constructing the at least one classifier comprises training the machine learning model using the fraction images captured at the first sorting arrangement and the fraction labels.

13. The method according to claim 12, wherein the feeding arrangement comprises a second sorting arrangement (10) arranged downstream to the first sorting arrangement (1 ), the method comprising: at the second sorting arrangement (10): capturing (S5) images of items in a stream of items (12); controlling (S6) sorting of items in the stream of items (12) into at least one sorted fraction (13) and optionally at least one unsorted fraction (14) using the images captured at the second sorting arrangement (10) and the at least one classifier.

14. The method according to any of claims 12-13, wherein labeling (S3) further comprises, for each fraction image: identifying at least one depicted item, such as by object instance segmentation, and wherein a respective fraction label is further indicative of each identified item.

15. The method according to claim 14, further comprising: extracting, from the set of fraction images, a set of fraction item images, each fraction item image depicting one identified item such that the number of fraction item images in the set of fraction item images is identical to a number of items identified in the set of fraction images; and wherein labeling (S3) is performed on the set of fraction item images, and wherein training the machine learning model is performed using the fraction item images captured at the first sorting arrangement and the fraction labels.

16. The method according to claim 14 or 15, further comprising: at the first sorting arrangement: capturing (S31 ) a set of stream images of the items in the stream of items (2) before sorting the stream of items (2) into at least one sorted fraction (3) and optionally at least one unsorted fraction (4); and labeling (S32) the stream images with stream labels, wherein labeling (S32) comprises, for each stream image, identifying at least one depicted item, wherein the stream label is further indicative of each identified item.

17. The method according to claim 16, wherein labeling (S32) further comprises: extracting, from the set of stream images, a set of stream item images, each stream item image depicting one identified item such that the number of stream item images in the set of stream item images is identical to a number of items identified in the set of stream images; and wherein labeling (S32) is performed on the set of stream item images.

18. The method according to claim 16 or 17, further comprising: comparing (S33) the set of stream images with the set of fraction images, or the set of stream item images with the set of fraction item images, and removing, from the set of stream images or stream item images, images depicting an item which is depicted in any one of thefraction images or fraction item images, thereby producing a modified set of stream images or a modified set of stream item images; labeling (S34) each stream image of the modified set of stream images, or each stream item image of the modified set of stream item images, with a label indicative of the depicted item having been sorted into the at least one unsorted fraction (4); and wherein constructing (S5) the at least one classifier further comprises training the machine learning model using the stream images of the modified set of stream images or the stream item images of the modified set of stream item images captured at the first sorting arrangement (1 ) and each associated label.

19. The method according to any one of claims 12-18, wherein labeling (S3) the fraction images captured at the first sorting arrangement (1 ) with fraction labels is performed by a processing unit (6), and wherein the processing unit (6) is further configured to provide the machine learning model with the fraction images captured at the first sorting arrangement and the fraction labels.

20. The method according to claim 18 and 19, wherein the steps according to claim 16 is performed by the processing unit (6), wherein the processing unit is further configured to provide the machine learning model with the stream images of the modified set of stream images or the stream item images of the modified set of stream item images captured at the first sorting arrangement (1 ) and each associated label.21 . The method according to any one of claims 12-20, further comprising distributing (S7) the at least one classifier from the first sorting arrangement (1 ) to the second sorting arrangement (10) via a network infrastructure, such as a cloud-based server.

22. The method according to claim 21 , further comprising updating (S8) the at least one classifier distributed to the second sorting arrangement (10), wherein updating the at least one classifier distributed to the second sorting arrangement (10) comprises: at the first sorting arrangement (1 ): repeating the steps of: sorting (S1 ) a stream of items into at least one sorted fraction (3) and optionally at least one unsorted fraction (4), capturing (S2) a set of fraction images of the items in the at least one sorted fraction (3), labeling (S3) the fraction images captured at the first sorting arrangement (1 ) with fraction labels, and constructing (S4) at least one classifier comprising training the machine learning model using the fraction images captured at the first sorting arrangement (1 ) and the fraction labels, thereby providing at least one updated classifier; distributing the at least one updated classifier to the second sorting arrangement (10); and activating the at least one updated classifier at the second sorting arrangement (10).

23. The method according to claim 22, wherein updating (S8) the at least one classifier distributed to the second sorting arrangement (10) further comprises sending a validation request from the second sorting arrangement (10) to the first sorting arrangement (1 ), wherein if the validation request is failed, performing the steps according to claim 10.

24. The method according to any of claims 12-23, wherein training the machine learning model comprises: embedding data of the fraction images captured at the first sorting arrangement (1 ) and the associated fraction labels and optionally data of the stream images captured at the first sorting arrangement (1 ) and the associated stream labels as data points in a spatial map, such as a 2- dimensional or a 3-dimensional map; and dividing the spatial map into cells, each cell having a data point densityand a date of last entry indicating when the latest data point was added to the cell.

25. The method according to claim 22 and 24, wherein repeating the step of constructing (S4) at least one classifier further comprises: for each cell, reviewing the data point density and / or the date of last entry, and if the data density of the cell is equal to or higher than a predetermined threshold, and / or if the date of last entry of the cell is equal to or older than a predetermined threshold, modifying a set of data points, such as removing every data point from the set of data points, contained within the cell.

26. The method according to claim 22 and 24, or claim 25, repeating the step of constructing (S4) at least one classifier further comprises: when adding a new data point to the spatial map, checking the data point density and / or the date of last entry of the cell into which the data point is to be added, and: if the data density of the cell is equal to or higher than a predetermined threshold, discarding the new data point; and / or if the date of last entry of the cell is equal to or older than a predetermined threshold, adding the new data point and removing at least one neighboring data point in the cell.

27. The method according to any one of claims 12-26, wherein constructing (S4) at least one classifier comprises constructing a respective classifier for each sorted fraction (3) of the at least one sorted fraction (3).

28. The method according to any one of the preceding claims, comprising:sharing the updated database with a second classifier associated with a second feeding arrangement.

29. The method according to any one of the preceding claims, comprising: updating the at least one classifier based at least on said dataset after said dataset has been updated.

30. The method according to any one of the preceding claims, comprising: sharing the updated classifier with a second arrangement, optionally via cloud communication.31 . The method according to any one of the preceding claims, wherein said dataset is a first dataset, and said feeding arrangement is a first feeding arrangement associated with a first application, such as waste processing or recycling, the method comprising: updating a second dataset based on an update in the first dataset, which second dataset is used by a classifier implemented in a second arrangement associated with a second application, wherein said second application is the same as the first application or different from said first application.

32. A computer readable storage medium having stored thereon instructions for implementing the method according to any one of claims 1 to 31.

Images