Method for detecting the size of objects in a waste pit

Abstract

A method for detecting objects based on size in a waste pit of a waste treatment plant, comprising the steps of: a) collecting images of waste in a waste pit of a waste treatment plant using a camera and transferring the images to a data processing unit; b) identifying individual objects in the images using a segmentation algorithm; c) determining the size of the individual identified objects; d) comparing the size of the individual identified waste objects with a threshold; and e) classifying whether or not the objects are bulky waste objects.

Description

[Technical Field]

[0001] The present invention relates to a method for detecting objects based on size within a waste pit in a waste treatment plant. [Background technology]

[0002] Waste is a reliable and important energy source in modern society. Thermal treatment of waste can reduce the environmental problems associated with the accumulation of waste in landfills. Waste thermal treatment is typically carried out in waste treatment plants that include waste pits and waste boilers. Since waste boilers have size limitations on the objects they can process, it is important to detect and separate objects that exceed certain size limits. Such objects are also known as "bulky waste objects." Typical bulky objects include mattresses, metal pipes, supermarket shopping carts, palm tree trunks, or trash cans.

[0003] Large waste objects can cause blockages in processing plants, leading to unplanned shutdowns. These shutdowns can result in costs of up to €100,000 per plant due to the loss of production capacity. Furthermore, the large objects causing the blockages must be removed, which poses health and safety risks to operators and often requires plant repair or restart.

[0004] In commonly used manual approaches, these bulky waste objects are detected by one or more operators, who often monitor up to eight screens simultaneously or visually inspect bunkers and label the bulky objects. To reduce the workload on operators, automated object detectors have been developed.

[0005] International Publication No. 2022185340 discloses a material detector for identifying waste, comprising a sensor, a detection unit, and a comparison unit. The sensor captures data from the outermost layer of the waste, and the detection unit determines one or more identification parameters of the waste material. The detection unit derives a digital fingerprint based on at least one physical and / or chemical parameter extracted from the data and compares the digital fingerprint with predetermined detectable physical and / or chemical parameters of the outermost layer stored in a product database. The comparison unit then labels the waste object based on the fingerprint and a relational table linking the waste object to an entry in the product database. This approach requires a database of all possible waste objects and their fragments, since waste may be supplied even partially. Therefore, a considerable amount of work is required to create and update this database in order to keep the waste detection functioning.

[0006] U.S. Patent Application Publication No. 2022 / 270238 discloses a system, apparatus, process, and method for measuring food, food consumption, and waste using image recognition. The system enables the measurement, classification, identification, and / or recording of food, food consumption, and food waste, and enables service providers, consumers, and other stakeholders to evaluate food consumption and waste over a period of time. [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] Therefore, an object of the present invention is to provide a method for detecting objects based on size within a waste pit of a waste treatment plant, which can improve the detection rate of bulky waste objects. [Means for solving the problem]

[0008] This objective is achieved by the method of the present invention described in claim 1 and the apparatus of the present invention described in claim 11. Preferred embodiments are the subject of the dependent claims.

[0009] According to claim 1, the present invention is a method for detecting objects based on size in a waste pit of a waste treatment plant, a) A step of using a camera to collect images of waste in a waste pit of a waste treatment plant and transferring the images to a data processing unit, b) A step of identifying individual objects in the image using a segmentation algorithm, c) A step of determining the size of each identified object, d) A step of comparing the size of each identified object with a threshold, e) A step of classifying whether or not the object is bulky waste, Regarding methods that include... [Modes for carrying out the invention]

[0010] In the context of the present invention, the term “waste pit” is used to describe a bunker of a waste treatment plant, including side walls and possible openings such as gates in the side walls.

[0011] In the context of this invention, the term "segment" refers to the output features of a segmentation algorithm. The collected images are processed by a segmentation algorithm, which creates (assigns) at least one segment for each object identified in the images. Therefore, the terms "segment" and "object" may be used interchangeably below.

[0012] Thanks to the use of a segmentation algorithm, the method of the present invention has the advantage of not requiring the time-consuming preparation of a database for training, as the method enables the determination of the size of an object regardless of its type or material. In other words, the method of the present invention does not aim to identify the type or nature of the waste object (e.g., "chair," "mattress," "plastic box," or "wood chip"), but simply to determine whether its size is above or below a given threshold. This reduction in information enables an efficient approach to identifying bulky waste objects. Therefore, identifying bulky waste objects is not a problem if the waste object is broken or if only a part of a particular object remains, as long as the visible portion or fragments of the original object exceed the threshold. This is a significant advantage because, unlike other prior art object detection techniques, it does not require increasing the complexity of the algorithm to enable the detection of partially covered objects. Furthermore, the threshold can be set based on the size limits of individual waste treatment plants.

[0013] According to the method of the present invention, step a) includes collecting images, which also covers collecting multiple images or a series of images.

[0014] With respect to the method of the present invention, it has been found that, surprisingly, despite a reduction in the amount of information to be processed or the number of steps to be performed, the detection rate of bulky waste objects can be significantly improved compared to methods of known art. In contrast to other prior art object detection techniques that are limited to detecting waste objects stored in a waste pit, the method of the present invention could also detect objects in the process of falling into a waste pit. Furthermore, the method of the present invention can also be used in waste treatment plants where waste is supplied to a waste pit from a delivery truck via a ramp or conveyor belt.

[0015] In a preferred embodiment of the present invention, step b) further includes identifying changes in the collected image compared to previously collected images of the same waste pit before identifying individual objects.

[0016] By comparing two images of the same waste pit collected at different times, it becomes possible to identify changes in the waste pit very quickly and accurately. These changes can be used to detect whether a truck has arrived and brought in new waste. These changes can also refer to newly transported waste objects in the waste pit or objects that change the position within the waste pit, for example, when a crane moves an object from one location to another within the waste pit. This additional comparison step is particularly useful when a large amount of waste is transported to the waste pit simultaneously, forming a new pile of waste, and as a result, not all of the newly transported objects are visible from the surface.

[0017] In the above embodiment, it is preferable that the time difference between two collected images is smaller than the feeding speed of the waste pit.

[0018] In other words, if waste is supplied to the waste pit by trucks and these trucks carry waste every three minutes, the time difference between two images is preferably less than three minutes. If the time difference between two images is smaller than or equal to the supply speed of the waste pit, it is ensured that all new batches of waste transported to the waste pit are scanned for bulky waste objects.

[0019] Generally, the time difference between two images is preferably less than one minute, and most preferably, the images are collected every second to ensure that all newly provided waste in the waste pit is detected and also for detection during falling if desired.

[0020] In a preferred embodiment of the present invention, a series of images are collected as the waste truck arrives and unloads the waste into the waste pit. This makes it possible to obtain one or more images of the waste in motion as it falls into the waste pit.

[0021] Preferably, the image is cropped to the region where the change is detected. In the context of this invention, the term "cropped" is used to describe a reduction in the geometric size (height and width) of the image. By cropping the image to the region where the change is detected, the further processing in steps c) and d) can be performed much more quickly.

[0022] In a preferred embodiment of the present invention, the images collected in step a) are modified using an algorithm for projection transformation before step b) (identification of individual objects) is performed. This modification is used to ensure that each pixel in the resulting projected image corresponds to a fixed measured length (for example, each pixel represents a length of 1 cm). This modification has the advantage of simplifying size estimation.

[0023] There are various methods for segmenting images. Some of the main techniques include semantic segmentation, instance segmentation, and panoptic segmentation. Common image segmentation techniques known in this art include edge-based segmentation, threshold-based segmentation, region-based segmentation, cluster-based segmentation, and watershed segmentation.

[0024] In a preferred embodiment of step b), the algorithm for segmentation is a graph-based segmentation algorithm, in particular the Felzenszwalb algorithm (also known as the Felzenszwalb-Huttenlocher algorithm) or an edge detection algorithm. Edge-based segmentation algorithms identify edges based on changes in contrast, texture, color, and saturation. These can accurately represent the boundaries of objects in an image using edge chains containing individual edges.

[0025] In the method of the present invention, the Felzenszwalb algorithm is particularly preferred because it has been found to be superior to other segmentation algorithms from the cluster or threshold families, as it is much faster than other segmentation algorithms with respect to the task of segmenting objects. Threshold segmentation algorithms use only two classes (0 or 1) and are therefore not very efficient in distinguishing boundaries between objects. On the other hand, clustering segmentation algorithms cluster only segments that have the same color, which is not suitable for size detection.

[0026] Preferably, an unsupervised algorithm is used for segmentation. In the context of this invention, the term “unsupervised algorithm” is used to describe an algorithm that does not require training samples to perform its task. In contrast, a supervised algorithm requires segmented images as training samples. The advantage of using an unsupervised algorithm is that it avoids the time-consuming task of labeling the training samples.

[0027] In a preferred embodiment of the present invention, the sizing of the identified individual objects in step c) includes fitting an ellipse to the segments of the identified individual objects, and then determining the centroid of the ellipse and the lengths of its major and minor axes, respectively. In this context, the major axis is the longest diameter of the ellipse, and the minor axis is the shortest diameter of the ellipse. The scikit-learn library in Python can be used to determine the centroid, major axis, and minor axis of the individual objects.

[0028] Surprisingly, the approach of estimating size using the center of gravity, major axis, and minor axis was found to be highly efficient and accurate.

[0029] Preferably, the lengths of the major and minor axes of the ellipse are compared to separate thresholds in step d).

[0030] In another preferred embodiment of the present invention, the sizing of the identified individual objects in step c) includes determining the length of the longest straight line between two points on the segment line.

[0031] In the context of this invention, the term "segment line" refers to the perimeter of a segment or object. Therefore, the longest straight line refers to the longest distance (straight line) between two points on the segment line.

[0032] Preferably, this length of the longest straight line is compared to a threshold in step d).

[0033] Both of the aforementioned preferred approaches—namely, comparing the lengths of the major and minor axes of an ellipse, and comparing the length of the longest straight line between two points on a segment line with a threshold—have been found to provide a good balance between accuracy and complexity in object detection. In particular, these 2D approaches have proven more useful than the more accurate 3D approaches using three vertical axes, which have been found to be much more computationally intensive and require more expensive hardware.

[0034] In a preferred embodiment of the present invention, the classification step e) is performed using a logistic regression model trained on a combination of features of the area of ​​individual objects, the ratio of area to perimeter, and / or shape index, in order to predict whether a segment corresponds to a bulky waste object.

[0035] Preferably, the combination of features includes either the area and the ratio of the area to the perimeter, or the area and the shape index, or the ratio of the area to the perimeter and the shape index. More preferably, the combination of features includes the area, the ratio of the area to the perimeter, and the shape index.

[0036] Surprisingly, the use of the logistic regression model described above was found to be a very accurate method for classifying whether an object is a bulky waste object or not. For example, to perform the classification step e), the previously trained logistic regression model uses the bulky objects previously identified in step d) as input to classify whether a bulky object is a bulky waste object or not.

[0037] Methods for calculating the area, the ratio of area to perimeter, and / or shape index as defined above are known to those skilled in the art.

[0038] Preferably, the area of ​​each object is calculated using the function `skiamge.measure.regionprops()` implemented in the Python library `scikit-image`.

[0039] Preferably, the ratio of the area to the perimeter of an individual object is calculated by dividing the area by the perimeter length, which is calculated using the function "skiamge.measure.regionprops()" implemented in the Python library "scikit-image".

[0040] The shape index of individual objects is preferably calculated using the `shape_index` function implemented in the Python library `scikit-image`. The shape index is a single-value measure of local curvature and is derived from the eigenvalues ​​of the Hessian matrix as defined by Koenderink & van Doorn.

[0041] In another preferred embodiment of the present invention, step e) classification is performed using a convolutional neural network (CNN) to classify whether an object is a bulky waste object. Surprisingly, the CNN was found to provide a very accurate classification of bulky waste objects. In particular, the use of a trained CNN has the advantage that the CNN has the ability to find underlying features during the training process, which eliminates the need for feature engineering.

[0042] Preferably, the CNN is trained on a training dataset containing images of waste and non-waste before being used in the classification step e). For example, the training dataset includes labeled images of typical waste objects such as mattresses, metal pipes, supermarket shopping carts, palm tree trunks, or trash cans, as well as labeled images of typical non-waste objects present in a waste pit, such as cranes, gates, and side walls.

[0043] In one preferred embodiment of the present invention, the image in step a) is collected by an RGB camera.

[0044] In a preferred embodiment of the present invention, the image in step a) is collected while the waste is falling into the waste pit. This has the advantage that individual objects can be more easily distinguished from one another in the falling motion compared to when the individual objects are stored as a pile of waste in the waste pit.

[0045] In a preferred embodiment of the present invention, step c) further includes identifying the material type of individual objects by using an identification algorithm trained on a training dataset containing labeled waste type materials.

[0046] The advantage of further identifying the material type of individual objects is that the waste pit will include not only waste objects that are too large to be incinerated, but also objects made from or containing materials that cannot be incinerated or should not be incinerated, such as metal pipes or gas bottles. If these objects can be divided into smaller or smaller parts, they will not be classified as bulky waste objects. Nevertheless, such objects should not be incinerated because they are not combustible.

[0047] Another aspect of the present invention relates to an apparatus for size-based object detection in a waste pit of a waste treatment plant, comprising a camera, a data processing unit, an image acquisition unit, an individual object identification unit, a size determination unit, and a classification unit. The image acquisition unit is configured to collect images of waste in the waste pit of the waste treatment plant. The individual object identification unit is configured to identify individual objects in the images using a segmentation algorithm. The size determination unit is configured to determine the size of the identified individual objects, and the classification unit is configured to compare the size of the identified individual objects with a threshold to classify whether or not the objects are bulky waste objects.

[0048] As described above in relation to the method of the present invention, the apparatus for object detection based on size in a waste pit of a waste treatment plant makes it possible to determine the size of an object regardless of its type or material. In other words, the apparatus of the present invention is not intended to identify the type or nature of waste (e.g., "chair," "mattress," "plastic box," or "wood chips"), but simply to determine whether its size is above or below a given threshold. This reduction in information allows the apparatus to implement an efficient approach to identifying bulky waste objects. Therefore, if the waste is broken or only a part of a particular object remains, it is not a problem to identify the bulky waste object using the apparatus according to the present invention, as long as the visible or broken part still exceeds the plant waste size limit defined by the threshold.

[0049] Preferably, the device uses an unsupervised algorithm for segmentation. This has the advantage of not requiring the time-consuming preparation of a database for training.

[0050] The present invention will be described below by non-limiting examples. In the waste pit of a waste treatment plant, an image I0 of the waste is collected by a camera at time T0. This image I0 may include a series of images. Image I0 is transferred to and stored in a data processing unit, in this case a computer. After image I0 is collected, a truck carries new waste to the waste pit and unloads the waste through an opening (gate) in the wall of the waste pit. After a batch of new waste has been delivered, a new image I1 of the waste is collected at time T1 and transferred to the data processing unit. The elapsed time between time T0 and T1 is preferably synchronized with the frequency of new waste delivery. This means that a new image is collected for each batch of newly delivered waste. Therefore, the time interval between two images can be variable. Alternatively, images can be collected with a fixed time interval between them. A variable time interval between images can be achieved, for example, with the help of a sensor that detects when new waste is delivered. This sensor can be the aforementioned camera, another camera, or a motion detection sensor that detects when a truck arrives. When the sensor detects such an arriving truck, all images collected thereafter are marked with a label (e.g., "Newly Transported Waste" or "Relevant") for further processing. When the sensor detects that the truck has left, all images collected thereafter are not marked with a label until the sensor detects the next arriving truck. This data processing unit stores only the images marked as relevant, along with the labels for further processing. This reduces computation time and saves memory space. It is also possible to collect a series of new images while new waste is being transported. In this case, the sensor activates the camera to capture a series of new images until the sensor is deactivated when the truck leaves. This series of images includes numerous images, each with its own individual timestamp, capturing the object (waste) as it falls from the truck into the waste pit.This is preferable because new waste loads may form a mountain in the waste pit, and if large objects are on the ground and covered by other newly transported waste objects, the large objects may not be detected. A series of images with individual timestamps also ensures that all large waste objects are detected when there are multiple large waste objects per truck. Therefore, collecting a series of new images can help detect objects that are later covered by the same truck or other waste transported in the same batch.

[0051] When the above sensors are installed, changes in the images can be detected according to the following non-limiting examples. Each newly collected image I X is compared with the previous image I X-1 and a difference in the images is detected. To detect such a difference, the images are first converted to grayscale and a region of interest mask is applied. For example, when the camera faces the transport gate of the waste pit, a region of interest mask with a polygon that matches the shape of each visible gate is applied. Such a mask sets all pixels outside the polygon, i.e., outside the gate, to black. Therefore, in the image to which the region of interest mask is applied, only the pixels related to a specific region (here the gate) contain visible content.

[0052] After this preprocessing is done for both images, image I X and I X-1 are compared at the pixel level, which means that the data processing unit uses an algorithm to compare all pixels of image I X with the corresponding pixels of image I X-1 , and pixels with no difference in images I X and I X-1 are excluded and changed to black to create a new image I ΔX . This process is also called image differentiation.

[0053] Image I ΔXAn erosion filter is applied to remove residual background noise, resulting in image I ΔXeroded Generates Image I ΔXeroded For comparison, the motion detector parameters are as follows: Image I ΔXeroded It is determined by calculating the standard deviation of the pixel values ​​in the image. If the motion detector parameter is below the threshold, it is presumed that no motion was detected, so image I ΔXeroded It will not be analyzed further. If the motion detector parameters exceed the threshold, image I X The bulky waste material is further analyzed to determine whether it has been transported to the waste pit.

[0054] Image I X This is further modified by a protective transformation commonly known as homography. In the protective transformation, Image I X The region arising from the plane (in this case, the gate) is the resulting image I. X-PT Each pixel within the image corresponds to a fixed distance measurement (1 cm in this case) to enable size estimation, as shown in Image I X-PT It is projected inward.

[0055] Image I X-PT The image is further processed using Felzenszwalb's algorithm for segmentation. Felzenszwalb's algorithm is a graph-based algorithm for segmenting an image into objects. This algorithm first places edges between each pair of adjacent pixels in the image, and these pixels are weighted according to features such as the difference in brightness and color between each adjacent pixel. Next, image segments are formed from each pixel and merged such that the difference between edge weights within the segment is as small as possible and between adjacent segments is as large as possible.

[0056] The identified segments (objects) are then treated as individual candidates for bulky waste objects in the waste pit. Each object is labeled with a tag containing a unique identifier.

[0057] For each identified object, an ellipse is fitted to the object, and the center of gravity, the length of the major axis, and the length of the minor axis are determined. The lengths of the major and minor axes are compared to separate thresholds. If the lengths of the major and minor axes exceed the thresholds, the object is considered a gross object and is further analyzed based on its geometric properties.

[0058] For each identified bulky object, the following geometric features are determined: the object's area, the ratio of the object's area to its perimeter, and its shape index. These features are then applied to a trained logistic regression model to determine whether the object is a bulky waste object or not.

Claims

1. A method for detecting objects based on size within a waste pit in a waste treatment plant, a) The steps of using a camera to collect images of waste in a waste pit of a waste treatment plant and transferring the images to a data processing unit, b) The step of identifying individual objects in the image using a segmentation algorithm, c) The step of determining the size of each identified object, d) A step of comparing the size of each identified object with a threshold, e) A step of classifying whether the object is bulky waste or not, A method that includes this.

2. The method according to claim 1, wherein the algorithm for segmentation in step b) is the Felzenszwalb algorithm or an edge detection algorithm.

3. The method according to either claim 1 or 2, wherein in step b), an unsupervised algorithm for segmentation is used.

4. The method according to any one of claims 1 to 3, wherein the size determination in step c) further comprises fitting an ellipse to the segments of the identified individual objects and determining the centroid of the ellipse and the lengths of the major and minor axes, respectively.

5. The method according to claim 4, wherein in step d), the lengths of the major axis and the minor axis are compared with separate thresholds.

6. The method according to any one of claims 1 to 3, wherein the size determination in step c) includes calculating the longest straight line between two points on the segment line of the object.

7. The method according to any one of claims 1 to 6, wherein step e) classifying is performed using a logistic regression model trained with features of area, area to perimeter, and / or shape index to classify whether or not an object is a bulky waste object.

8. The method according to any one of claims 1 to 6, wherein step e) classifying is performed using a convolutional neural network to classify whether or not an object is a bulky waste object.

9. The method according to any one of claims 1 to 8, wherein the image in step a) is collected using an RGB camera.

10. The method according to any one of claims 1 to 9, wherein the image of step a) is collected while the waste is falling into the waste pit.

11. A device for detecting objects based on size within a waste pit in a waste treatment plant, Camera and, Data processing unit, An image acquisition unit configured to collect images of waste in a waste pit of a waste treatment plant, An individual object identification unit configured to identify individual objects in the image using a segmentation algorithm, A size determination unit configured to determine the size of each identified object, A classification unit configured to compare the size of each identified waste object with a threshold and classify whether or not the object is a bulky waste object, A device equipped with the following features.

Images