Intelligent garbage classification method based on computer vision

Abstract

The present application relates to the field of garbage classification and recycling, and particularly relates to a garbage intelligent classification method based on computer vision. A garbage grayscale image after complete paving is collected; uniformity of each pixel point is obtained; adhesion of each pixel point is obtained according to average distance between each pixel point and other pixel points with same gradient amplitude and the uniformity of each pixel point; a filter window size of each pixel point is obtained, each pixel point in the garbage grayscale image is filtered, and a filtered garbage grayscale image is obtained; plastic bags in each connected domain are recognized by using a neural network, and the plastic bags in the garbage are separated. The present application selects an adaptive filter window for the garbage grayscale image, thereby recognizing the plastic bags, can accurately separate the plastic bags in the garbage, and effectively improves the efficiency of garbage classification.

Description

Technical Field

[0001] This invention relates to the field of waste sorting and recycling, and specifically to a computer vision-based intelligent waste sorting method. Background Technology

[0002] With rapid societal development, various products are constantly emerging, using a wide variety of materials, resulting in increasing amounts of waste and serious environmental pollution problems. As early as the last century, people recognized the importance of waste sorting. With advancements in science and technology, waste sorting has become increasingly intelligent. Currently, waste sorting mainly relies on people's traditional understanding for initial classification. However, due to habit or the need to wrap some waste, some plastic bags still remain in the initially sorted waste. This necessitates secondary sorting when the waste is transported to the processing station. During secondary sorting, the plastic bags wrapped around some of the waste need to be broken open to separate them from the waste. At this point, the plastic bags must be picked out from the waste.

[0003] Existing technologies include air separation to separate plastic bags from garbage. However, this process may require multiple air separations to ensure complete separation of plastic bags from the garbage. Furthermore, air separation results in significant odor emission, low efficiency, and a low level of intelligence. Methods using computer vision to identify plastic bags in garbage face challenges. Due to the complex environment in which plastic bags and garbage are located, the images obtained through computer vision contain a large amount of noise. Directly performing edge detection on the image and then using a neural network for plastic bag identification can lead to misidentification of garbage as plastic bags, resulting in inaccurate plastic bag identification. Consequently, when separating plastic bags, similar garbage in the image may be picked up, leading to poor garbage sorting results. Summary of the Invention

[0004] To address the problem of high noise levels in existing computer vision images, which leads to inaccurate plastic bag identification, this invention provides a computer vision-based intelligent waste sorting method. The method includes: acquiring a fully tiled grayscale image of waste; obtaining the uniformity of each pixel; determining the adhesion of each pixel based on the average distance between each pixel and other pixels with the same gradient magnitude, as well as the uniformity of each pixel; obtaining the filtering window size for each pixel; filtering each pixel in the grayscale image to obtain a filtered grayscale image; and using a neural network to identify plastic bags in each connected component, thus separating the plastic bags from the waste. This invention selects an adaptive filtering window for the grayscale image of waste to identify plastic bags, accurately separating them from the waste and effectively improving the efficiency of waste sorting.

[0005] This invention adopts the following technical solution: a computer vision-based intelligent waste sorting method, comprising:

[0006] The plastic bag wrapping the garbage is torn open using a roller device. The garbage and the torn plastic bag are then laid flat on a conveyor belt. A grayscale image of the garbage after it is completely laid flat on the conveyor belt is then captured.

[0007] The uniformity of each pixel is obtained by considering the gradient magnitude of each pixel in the grayscale image and each pixel in its eight neighborhoods.

[0008] Obtain the distance between pairs of pixels with the same gradient magnitude in the garbage grayscale image, and obtain the adhesion degree of each pixel based on the average distance between each pixel and other pixels with the same gradient magnitude and the uniformity of each pixel;

[0009] The size of the filtering window for each pixel is obtained based on the adhesion of each pixel. Each pixel in the garbage grayscale image is filtered using the size of the filtering window for each pixel to obtain the filtered garbage grayscale image.

[0010] Edge detection is performed on the filtered grayscale image of the garbage to obtain multiple connected components. The filtered grayscale image of the garbage is used as the input of a neural network, and the output is the plastic bags in each connected component. The plastic bags in each connected component are then separated from the garbage.

[0011] Furthermore, a computer vision-based intelligent waste classification method obtains the uniformity of each pixel as follows:

[0012] Obtain the gradient magnitude of each pixel in a grayscale image of garbage;

[0013] The information entropy of each pixel is obtained by calculating the gradient magnitude of each pixel and its eight neighboring pixels.

[0014] Obtain the mean value of the gradient magnitude difference between each pixel and its eight neighboring pixels;

[0015] The uniformity of each pixel is obtained by taking the average of the gradient magnitude differences between each pixel and its eight neighboring pixels, as well as the information entropy of that pixel.

[0016] Furthermore, a computer vision-based intelligent waste classification method obtains the adhesion degree of each pixel as follows:

[0017] Obtain the average distance between each pixel in a garbage grayscale image and its pixels with the same gradient.

[0018] Get the gradient magnitude of each pixel and the ratio of the average distance between each pixel and the pixels with the same gradient.

[0019] The adhesion degree of each pixel is obtained by multiplying this ratio by the uniformity of each pixel in the garbage grayscale image.

[0020] Furthermore, a computer vision-based intelligent waste classification method uses the following expression to obtain the adhesion degree of each pixel:

[0021]

[0022] Among them, F i d represents the adhesion of the i-th pixel. i G represents the average distance between the i-th pixel and other pixels with the same gradient. i Let J represent the gradient magnitude of the i-th pixel, e represent the exponential function with base e, and J represent the gradient magnitude of the i-th pixel. i This represents the uniformity of the i-th pixel.

[0023] Furthermore, a computer vision-based intelligent waste classification method obtains the filter window size for each pixel as follows:

[0024] The adhesion of each pixel is used as the exponent of the exponential function;

[0025] The product of the exponential function and the set constant is obtained and rounded up to get the filter window size for each pixel.

[0026] Furthermore, a computer vision-based intelligent waste sorting method uses a filtered grayscale image of the waste as input to a neural network, and outputs the plastic bags in each connected component as follows:

[0027] We obtained grayscale images of plastic bags of various colors as a dataset and used the dataset to train the neural network.

[0028] By using a trained neural network to identify targets in multiple connected components of a filtered grayscale image of garbage, the plastic bags in the filtered grayscale image of garbage are obtained.

[0029] Furthermore, a computer vision-based intelligent waste classification method involves filtering each pixel in a grayscale waste image as follows:

[0030] Using each pixel in the grayscale image as the center point, the Gaussian kernel of the filtering window where each center point is located is obtained based on the grayscale values ​​of all pixels in the filtering window where each center point is located.

[0031] Gaussian filtering is performed on each center point based on the Gaussian kernel of the filtering window in which each center point is located.

[0032] The beneficial effects of this invention are as follows: This invention reflects the distribution of pixels in an image by obtaining the uniformity of each pixel, which can be used as one of the indicators to distinguish pixels from noise in an image. Furthermore, by combining the average distance between each pixel and pixels with the same gradient, the adhesion of each pixel can be obtained, which can reflect the gray-level changes in the area where each pixel is located and the degree of pixel aggregation. Based on this, the filtering window of each pixel is adaptively obtained, ensuring a good filtering effect on the pixels in the image. This can effectively improve the accuracy of subsequent plastic bag recognition and help to realize intelligent waste classification. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of a computer vision-based intelligent waste sorting method according to an embodiment of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] like Figure 1 The diagram shows a structural schematic of a computer vision-based intelligent waste sorting method according to an embodiment of the present invention, including:

[0037] 101. Collect a grayscale image of the garbage after it has been fully tiled.

[0038] The scenario addressed by this invention is as follows: When transporting sorted kitchen waste from a community to a recycling station for secondary processing, the kitchen waste on the conveyor belt needs to be further sorted to separate the plastic bags. Therefore, the prerequisite for collecting grayscale images of the waste in this invention is that a roller device is installed at the feed inlet of the conveyor belt. This roller device is equipped with serrated or steel-blade-like sharp blades to break open the plastic bags in the waste, allowing the waste inside to be poured out. Secondly, a shaking device is installed in the conveyor belt to shake the kitchen waste on the conveyor belt into a flat surface, ensuring that the kitchen waste does not pile up. At this time, a high-definition camera installed above the conveyor belt collects images of the flattened waste on the conveyor belt. When the conveyor belt is in the open state, this invention needs to collect one image of the waste on the conveyor belt at regular intervals. Assuming that the conveyor belt moves at a uniform linear speed of v, the width of the conveyor belt is l, and the aspect ratio of the collected image is 4:3, then the sampling interval t is:

[0039]

[0040] To prevent the same object from being captured in different frames during image acquisition, sampling needs to be performed within a short sampling time interval, t′ = t - α. To reduce the impact of light on subsequent image processing, the light source during image acquisition needs to shine evenly on the surface of the conveyor belt from top to bottom, and the image is captured by a high-magnification camera.

[0041] The acquired image is then converted to grayscale to obtain the corresponding grayscale garbage image. The specific grayscale conversion method is existing technology and is not limited in this invention.

[0042] Because the edges of the detected kitchen waste contain a lot of edge noise, it is necessary to perform noise reduction processing on the edges in order to prevent noise from affecting the obtained virtual edges when acquiring the edges.

[0043] Kitchen waste mainly includes discarded vegetable leaves, leftover food, fruit peels, eggshells, tea dregs, bones, etc. Plastic bags in kitchen waste have relatively obvious characteristics compared to other kitchen waste. However, the shapes and types of waste are numerous, and there are many interfering factors when identifying them. For example, large vegetable leaves and fruit peels can affect the identification of plastic bags. After obtaining the grayscale image of the waste, this invention further uses the Canny edge detection algorithm to obtain the edge pixels in the grayscale image of the waste.

[0044] 102. Obtain the uniformity of each pixel in the garbage grayscale image based on the gradient magnitude of each pixel in its eight neighborhoods.

[0045] When performing noise reduction on an image, it is necessary to consider the degree of influence of noise on edge pixels during edge detection. This is because noise pixels are randomly distributed, while kitchen waste, although having many morphological features, can be considered equal or similar in grayscale features within a very small range. Therefore, this invention distinguishes noise from the original pixels by calculating the uniformity of pixels within a small range.

[0046] The method for obtaining the uniformity of each pixel is as follows:

[0047] Obtain the gradient magnitude of each pixel in a grayscale image of garbage;

[0048] This invention first calculates the gradient continuity of each pixel to obtain the uniformity of the pixels. The gradient magnitude and gradient direction of each pixel are calculated as follows:

[0049]

[0050]

[0051] In the formula, G(x,y) represents the gradient magnitude, θ represents the gradient direction, and g x Let g represent the partial derivative of the bivariate function f(x,y) with respect to x. y Let f(x,y) represent the partial derivative of a bivariate function f(x,y) with respect to y. The gradient direction takes the absolute value, therefore the resulting angle range is [0, y]. 0 180 0 At each pixel, the gradient has a magnitude and a direction. The gradient in the x-direction strengthens vertical edge features, and the gradient in the y-direction strengthens horizontal edge features.

[0052] The information entropy of each pixel is obtained by calculating the gradient magnitude of each pixel and its eight neighboring pixels.

[0053] Obtain the mean value of the gradient magnitude difference between each pixel and its eight neighboring pixels;

[0054] The uniformity of each pixel is obtained by using the mean of the gradient magnitude differences between each pixel and its eight neighboring pixels, as well as the information entropy of that pixel. The expression is:

[0055]

[0056] In the formula, J i G represents the uniformity of the i-th pixel. j (x,y) represents the gradient magnitude of the j-th pixel within the eight neighborhood of the i-th pixel at coordinates (x,y), and G(x,y) represents the gradient magnitude at the i-th pixel as the center point. in This represents the proportion of the gradient magnitude of the nth pixel within the eight neighborhood of the i-th pixel. Let represent the information entropy within the eight neighborhoods of the i-th pixel, and let represent the disorder of the gradient within the eight neighborhoods of each pixel.

[0057] The gradient value of a pixel represents the change in the grayscale value of the pixel. By calculating the average gradient within the eight neighborhoods of each pixel, we can reflect the overall gradient magnitude of the surrounding pixels. The information entropy of the gradient is calculated to represent the degree of gradient change within the template. If the change in gradient information entropy is large, it indicates that the grayscale value of the pixels within the template changes drastically, indicating that its uniformity is low.

[0058] 103. Obtain the distance between pairs of pixels with the same gradient magnitude in the garbage grayscale image, and obtain the adhesion degree of each pixel based on the average distance between each pixel and other pixels with the same gradient magnitude and the uniformity of each pixel.

[0059] In the process of denoising a grayscale image with junk pixels, noise pixels are redundant pixels relative to the junk pixels in the image. Moreover, noise pixels are mostly discrete and messy pixels. Compared with the pixels in the junk area of ​​the image itself, the degree of pixel dispersion is greater. The adhesion degree represents the aggregation of pixels in this area and the change in grayscale. Therefore, the adhesion degree of noise pixels is relatively small.

[0060] The method for obtaining the adhesion of each pixel is as follows:

[0061] Obtain the average distance between each pixel in a garbage grayscale image and its pixels with the same gradient.

[0062] The adhesion degree of each pixel is obtained based on the uniformity of each pixel in the grayscale image, the gradient magnitude of each pixel, and the average distance between each pixel and its pixel with the same gradient.

[0063] The expression for obtaining the adhesion of each pixel is:

[0064]

[0065] Among them, F i d represents the adhesion of the i-th pixel. i G represents the average distance between the i-th pixel and other pixels with the same gradient. i Let J represent the gradient magnitude of the i-th pixel, e represent the exponential function with base e, and J represent the gradient magnitude of the i-th pixel. i This represents the uniformity of the i-th pixel.

[0066] Since the gray values ​​of noise pixels differ from those of pixels in the original image, and the distribution of noise is quite chaotic, this invention represents the feature value of a pixel based on its gradient magnitude and the average distance between pixels. The larger the gradient magnitude and the farther the average distance from other pixels with the same gradient, the greater the gray value change of the pixel, which means that it is more likely to be a noise pixel. Furthermore, the smaller the uniformity of a pixel, the more complex the gray value changes of pixels in its neighborhood, which also means that it is more likely to be a noise pixel. Therefore, the smaller the pixel's adhesion, the greater the likelihood that the pixel is noise. Subsequently, a filtering window is selected for filtering each pixel based on its adhesion.

[0067] 104. Obtain the filter window size for each pixel based on the adhesion of each pixel, and filter each pixel in the garbage grayscale image using the filter window size of each pixel to obtain the filtered garbage grayscale image.

[0068] The method for obtaining the filter window size for each pixel is as follows:

[0069] The adhesion of each pixel is used as the exponent of the exponential function;

[0070] The product of the exponential function and the set constant is taken and rounded up to obtain the filter window size for each pixel. The expression is:

[0071]

[0072] Among them, K i Let F represent the size of the filtering window for the i-th pixel, e represent the exponential function with base e, and F... i This represents the adhesion degree of the i-th pixel. Since the filter window should be a positive integer, a constant of 10 is set and multiplied by the exponential function. The result of the multiplication is then rounded up to obtain the accurate filter window size.

[0073] The smaller the adhesion, the more likely the pixel is to be noise. Therefore, the larger the filtering window for filtering it, the smaller the weight of the center pixel can be when performing Gaussian filtering, thus achieving a better noise reduction effect.

[0074] The method for filtering each pixel in a grayscale image of garbage is as follows:

[0075] Using each pixel in the grayscale image as the center point, the Gaussian kernel of the filtering window where each center point is located is obtained based on the grayscale values ​​of all pixels in the filtering window where each center point is located.

[0076] Gaussian filtering is performed on each center point based on the Gaussian kernel of the filtering window in which each center point is located.

[0077] Based on the filtering window and grayscale value of each pixel, a Gaussian kernel is generated using a Gaussian function:

[0078]

[0079] In the formula, K i Let f(x,y) be the size of the filtering window, f(x,y) represent the gray value of pixel (x,y), and σ be the standard deviation. The Gaussian kernel standard deviation σ is obtained from the variance. The formula for calculating the variance of a certain region of the image is:

[0080]

[0081] in:

[0082]

[0083] In the formula, S represents the average gray value of the pixels in the convolution window within the region containing the center point. x,y Let f(m,n) represent the size of the convolution window containing the center point (x,y), and let f(m,n) represent the grayscale value of the pixel at coordinates (m,n) within the convolution window. N x,y This represents the number of pixels contained in the convolution window of the region where the center point (x,y) is located. The larger the variance D(i,j), the greater the S x,y The greater the dispersion of pixel values ​​within a region, the greater the weight of the Gaussian kernel coefficients generated by selecting a smaller σ, and the smaller the impact on that region.

[0084] The specific operation of Gaussian filtering in this invention is as follows: scan each pixel in the image with a template (or convolution, mask), and replace the value of the center pixel of the template with the weighted average gray value of the pixels in the neighborhood determined by the template. This is the filtering process of Gaussian filtering in the prior art, which will not be elaborated on in this invention.

[0085] 105. Perform edge detection on the filtered grayscale image of the garbage to obtain multiple connected components. Use a neural network to identify the plastic bags in each connected component and separate the plastic bags.

[0086] The method for identifying plastic bags in each connected component using a neural network is as follows:

[0087] We obtained grayscale images of plastic bags of various colors as a dataset and used the dataset to train the neural network.

[0088] The trained neural network is used to identify targets in multiple connected components of the filtered grayscale image of garbage, thereby obtaining the plastic bags in the filtered grayscale image of garbage.

[0089] Because garbage bags are all made of plastic and their colors are relatively uniform, making them easy to identify, this invention trains a CNN neural network to learn the colors of garbage bags. The input consists of plastic garbage bags of various colors. The trained CNN neural network then uses the denoised grayscale image of the garbage to identify the plastic bags, thereby picking up the plastic bags from the conveyor belt. This removes the impact of the plastic bags on the secondary sorting of garbage. The remaining garbage on the conveyor belt is then further sorted according to different garbage types, completing the intelligent sorting of garbage.

[0090] This invention reflects the distribution of pixels in an image by obtaining the uniformity of each pixel, which can be used as one of the indicators to distinguish pixels from noise in an image. Furthermore, it combines the average distance between each pixel and pixels with the same gradient to obtain the adhesion of each pixel, which can reflect the gray-level changes in the area where each pixel is located and the degree of pixel aggregation. Based on this, the filtering window of each pixel is adaptively obtained to ensure a good filtering effect on the pixels in the image, which can effectively improve the accuracy of subsequent plastic bag recognition and help to realize intelligent waste classification.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A computer vision-based intelligent waste sorting method, characterized in that, include The plastic bag wrapping the garbage is torn open using a roller device. The garbage and the torn plastic bag are then laid flat on a conveyor belt. A grayscale image of the garbage after it is completely laid flat on the conveyor belt is then captured. The uniformity of each pixel is obtained by considering the gradient magnitude of each pixel in the grayscale image and each pixel in its eight neighborhoods. Obtain the distance between pairs of pixels with the same gradient magnitude in the garbage grayscale image, and obtain the adhesion degree of each pixel based on the average distance between each pixel and other pixels with the same gradient magnitude and the uniformity of each pixel; The size of the filtering window for each pixel is obtained based on the adhesion of each pixel. Each pixel in the garbage grayscale image is filtered using the size of the filtering window for each pixel to obtain the filtered garbage grayscale image. Edge detection is performed on the filtered grayscale image of the garbage to obtain multiple connected components. The filtered grayscale image of the garbage is used as the input of the neural network, and the output is the plastic bags in each connected component. Separate plastic bags from garbage in each connected domain.

2. The intelligent waste sorting method based on computer vision according to claim 1, characterized in that, The method for obtaining the uniformity of each pixel is as follows: Obtain the gradient magnitude of each pixel in a grayscale image of garbage; The information entropy of each pixel is obtained by calculating the gradient magnitude of each pixel and its eight neighboring pixels. Obtain the mean value of the gradient magnitude difference between each pixel and its eight neighboring pixels; The uniformity of each pixel is obtained by taking the average of the gradient magnitude differences between each pixel and its eight neighboring pixels, as well as the information entropy of that pixel.

3. The intelligent waste sorting method based on computer vision according to claim 1, characterized in that, The method for obtaining the adhesion of each pixel is as follows: Obtain the average distance between each pixel in a garbage grayscale image and its pixels with the same gradient. Get the gradient magnitude of each pixel and the ratio of the average distance between each pixel and the pixels with the same gradient. The adhesion degree of each pixel is obtained by multiplying this ratio by the uniformity of each pixel in the garbage grayscale image.

4. The intelligent waste sorting method based on computer vision according to claim 3, characterized in that, The expression for obtaining the adhesion of each pixel is: Among them, F i d represents the adhesion of the i-th pixel. i G represents the average distance between the i-th pixel and other pixels with the same gradient. i Let J represent the gradient magnitude of the i-th pixel, e represent the exponential function with base e, and J represent the gradient magnitude of the i-th pixel. i This represents the uniformity of the i-th pixel.

5. The intelligent waste sorting method based on computer vision according to claim 1, characterized in that, The method for obtaining the filter window size for each pixel is as follows: The adhesion of each pixel is used as the exponent of the exponential function; The product of the exponential function and the set constant is obtained and rounded up to get the filter window size for each pixel.

6. The intelligent waste sorting method based on computer vision according to claim 1, characterized in that, The method of using the filtered grayscale image of the garbage as input to a neural network and outputting the plastic bags in each connected component is as follows: We obtained grayscale images of plastic bags of various colors as a dataset and used the dataset to train the neural network. By using a trained neural network to identify targets in multiple connected components of a filtered grayscale image of garbage, the plastic bags in the filtered grayscale image of garbage are obtained.

7. The intelligent waste sorting method based on computer vision according to claim 1, characterized in that, The method for filtering each pixel in a grayscale image of garbage is as follows: Using each pixel in the grayscale image as the center point, the Gaussian kernel of the filtering window where each center point is located is obtained based on the grayscale values ​​of all pixels in the filtering window where each center point is located. Gaussian filtering is performed on each center point based on the Gaussian kernel of the filtering window in which each center point is located.

Images