A waste plastic pose recognition method based on machine vision

Abstract

The application discloses a waste plastic pose recognition method based on machine vision, which comprises the following steps: acquiring the image of waste plastic through an industrial color camera; performing gray processing on the acquired RGB image according to a gray-scale formula; dividing the image into multiple horizontal regions and performing threshold segmentation of limited experience gray levels on each channel; extracting the contour of waste plastic in each channel and calculating the channel centroid; finally, calculating the centroid and angle of the whole waste plastic according to the centroid coordinates of multiple channels to obtain the pose of the waste plastic. The method increases the calculation of angle information, thereby increasing the success rate of large-angle waste plastic sorting; and reduces the traversal range of the best threshold when performing threshold segmentation, thereby reducing the calculation amount of threshold segmentation and the time consumed by image recognition.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and more specifically to a method for recognizing the pose of waste plastics based on machine vision. Background Technology

[0002] The beverage industry extensively uses lightweight plastic bottles as packaging containers. These bottles are made from non-renewable petroleum through polymerization and processing, making them difficult to degrade naturally. If waste plastics are not recycled, the resulting waste, whether landfilled, incinerated, or randomly dumped, will cause significant harm to the ecological environment and result in substantial resource waste. Sorting and recycling waste plastics is key to solving this problem. Waste plastics vary in color, quality, and size during recycling, necessitating sorting. Currently, waste plastic sorting both domestically and internationally is primarily done manually. Manual sorting is not only labor-intensive and inefficient but also creates a poor working environment. Therefore, there is an urgent need for a fast and stable automated sorting system to replace manual sorting.

[0003] Machine vision systems can capture, process, and analyze images at extremely high speeds, improving processing efficiency and capabilities. They can handle different types and shapes of waste plastics and can seamlessly integrate with other equipment such as conveyor belts and pneumatic devices to achieve fully automated waste plastic sorting production lines. Generally, the nozzles in the pneumatic sorting section at the end of a waste plastic sorting system are discrete, and the centroid of the plastic often does not match the position of the nozzle. When sorting waste plastics with large shapes and orientations, insufficient air pressure and flow rate from a single nozzle can lead to sorting failure. Therefore, waste plastic sorting may require multiple nozzles to spray air at staggered times, which necessitates sensing not only the centroid position of the waste plastic but also its orientation information. Below are some research advances in position and orientation recognition algorithms.

[0004] Patent CN109190493A discloses an image recognition method that uses grayscale processing to obtain a grayscale image, and then performs multi-threshold search on the grayscale image to obtain the optimal harmony threshold solution set for segmentation, which can accurately locate the specific location of the apple. Although this method accurately locates the apple's position using threshold segmentation, it does not calculate its posture and is not suitable for sorting waste plastics with large angles.

[0005] Patent CN115082560A discloses a material pose recognition method that acquires an original image containing a part, generates the minimum bounding rectangle of the part in the original image, obtains the part's position data and principal direction angle based on the minimum bounding rectangle, and maps the position data and rotation angle to the world coordinate system to obtain the part's pose. Patent CN113269835A provides an industrial part pose recognition method based on contour features. It acquires an image of the industrial part, then obtains the coordinates of the edge pixels of the industrial part based on the image, then obtains the coordinates of the four corner points of the minimum bounding rectangle of the industrial part based on the edge pixel coordinates, and finally calculates the coordinates of the center point of the minimum bounding rectangle as the position coordinates of the industrial part, and then calculates the pose angle of the industrial part. Both patents use the minimum bounding rectangle method to calculate the position and angle information of the part. However, experiments show that when using the minimum bounding rectangle method for threshold segmentation and traversing the optimal threshold, it is necessary to calculate the information entropy of 256 gray levels, which places high demands on the computer hardware system. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a machine vision-based method for recognizing the pose of waste plastics. This method can quickly identify the pose of waste plastics on a conveyor belt, calculate the position information and angle information of the waste plastics, thereby reducing the time consumed by the algorithm and improving the sorting effect of waste plastics at large angles.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] A machine vision-based method for recognizing the pose of waste plastics includes the following steps:

[0009] Step S1: Install the industrial color camera above the conveyor belt. When waste plastic appears within the field of view of the industrial color camera, the industrial color camera will capture images of the waste plastic in real time.

[0010] Step S2: Use a weighted average method to convert the image obtained in step S1 to grayscale to distinguish the waste plastic from the background in the original image;

[0011] Step S3: The grayscale image obtained in step S2 is segmented into multiple horizontal regions corresponding to the camera's field of view using the horizontal average cutting method. Then, a fixed range of thresholds is set for each region to perform empirical threshold segmentation. After threshold segmentation, a binarized image is obtained.

[0012] Step S4: The Suzuki algorithm is used to extract contours from multiple regions of the binarized image obtained by threshold segmentation in step S3. Then, the position of the centroid of multiple regions is calculated using contour moments. Finally, the centroid and angle of the waste plastic as a whole are calculated according to the centroid position of each region, thus realizing the pose recognition of waste plastic.

[0013] The beneficial effects of this invention are as follows: The empirical threshold segmentation used in this invention divides the image into N horizontal regions corresponding to the camera's field of view by horizontal cutting, and then performs empirical threshold segmentation. The angle of the overall waste plastic is calculated using the centroid position of each region. This allows for precise control over the number of nozzles used and the time of each nozzle's action when sorting waste plastic at large angles, and also reduces the algorithm's calculation time, thereby improving the efficiency and success rate of waste plastic sorting. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention.

[0015] Figure 2 The flowchart shows the calculation process for the empirical threshold segmentation of different regions.

[0016] Figure 3 This is the original image of waste plastic.

[0017] Figure 4 This is a grayscale image of waste plastic.

[0018] Figure 5 This is a grayscale image of waste plastic after it has been segmented by region.

[0019] Figure 6 Topological relationship diagram of the contour boundary.

[0020] Figure 7 This is the result image after contour extraction of waste plastics by region. Detailed Implementation

[0021] The present invention will be described in detail below with reference to the accompanying drawings and examples. The purpose and effects of the present invention will become clearer. The specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0022] like Figure 1 and Figure 2 As shown, this invention is a pose recognition method for real-time sorting of waste plastics, specifically including the following steps:

[0023] Step S1: Install the industrial color camera above the conveyor belt. When waste plastic appears within the field of view of the industrial color camera, the industrial color camera will acquire an image of the waste plastic in real time and transmit it to the computer.

[0024] Step S2: Use a weighted average method to convert the image obtained in step S1 to grayscale to distinguish the waste plastic from the background in the original image;

[0025] Step S3: The grayscale image obtained in step S2 is divided into N horizontal regions corresponding to the camera's field of view using the horizontal average cutting method. Then, a fixed range of thresholds is set for each region to perform empirical threshold segmentation. After threshold segmentation, a binarized image is obtained.

[0026] Step S4: The Suzuki algorithm is used to extract the contours of N regions in the binarized image obtained after the region-based empirical threshold segmentation in Step S3. Then, the position of the centroid of these N regions is calculated using the contour moments. Finally, the centroid and angle of the waste plastic as a whole are calculated according to the centroid position of each region, thus realizing the pose recognition of waste plastic.

[0027] The method for recognizing the pose of waste plastic based on machine vision, specifically step S2, involves multiplying the color values ​​of the three RGB channels of the image acquired by the industrial color camera by different weights according to their importance, and then summing them to obtain a grayscale value. This process converts the RGB image into a grayscale image. The specific calculation formula is shown in formula (1).

[0028] Grey=αR+βG+γB#(1)

[0029] α, β, and γ are weights derived from the brightness contributions of the three different color channels. Experimental comparisons demonstrated that α = 0.07, β = 0.72, and γ = 0.21 yielded the best grayscale processing results, as shown below. Figure 3 and Figure 4 These are the original color images of waste plastic and the resulting images after grayscale processing.

[0030] The method for recognizing the pose of waste plastic based on machine vision, specifically step S3, involves: To shorten the time required to find the segmentation threshold during the threshold segmentation process, the segmentation threshold is determined in advance instead of iterating through each frame of the image. This fixed threshold segmentation method is suitable for situations where the background is relatively simple and the segmentation threshold fluctuates little, which aligns well with the working scenario of this invention. In this invention, the black-gray conveyor belt background exhibits minimal local pixel value changes during movement, but significant color differences exist between different regions. In a grayscale mode ranging from 0 to 255, the grayscale value difference reaches approximately 50. Using the same fixed threshold for segmentation across different regions would result in substantial errors. Therefore, the image captured by the camera is divided into N regions, with a fixed empirical threshold set for each region. The height of each region is calculated based on the image height, and then the image is horizontally segmented according to a predefined number of rows. The N segmented strip regions can be statistically analyzed for their empirical average grayscale value as the basis for calculating the fixed threshold for that region.

[0031] Taking N=12 as an example, Figure 5 The result of grayscale image segmentation is shown in Table 1 below. Some of the test data are listed in Table 1. The values ​​in Table 1 represent the average grayscale value of the channel when there is only background. Blank spaces indicate that there is plastic to be identified in the data point and therefore it is not included in the calculation.

[0032] Table 1. Statistics of Average Gray Values ​​for Each Channel

[0033]

[0034]

[0035]

[0036] The empirical grayscale values ​​of the background for each channel when N=12 were calculated from the data in Table 1, and the results are shown in Table 2 below. The empirical grayscale average value in Table 2, plus a specific adjustment value, with the empirical adjustment value set to 50, can be used as the maximum entropy threshold traversal range for distinguishing the background and foreground.

[0037] Table 2. Average grayscale values ​​for each channel.

[0038]

[0039] When performing thresholding on an image, the entire image is first divided into N horizontal regions. Then, each region is individually segmented using maximum entropy thresholding. Assuming the average background grayscale value of a channel is x, the range of the maximum entropy segmentation threshold is not [0, 255] but [x, x+50]. If the threshold that maximizes the image entropy occurs between x and x+50, then that threshold is the maximum entropy segmentation threshold. If the image entropy monotonically increases within the range [x, x+50], then x+50 is selected as the segmentation threshold. Although this threshold is not the maximum entropy segmentation threshold for the entire grayscale range [0, 255], it is sufficient to distinguish the plastic and the conveyor belt background in the image. Therefore, it is not necessary to continue searching for the maximum entropy segmentation threshold. In this way, the range of segmentation thresholds for each channel is reduced from 256 values ​​(0-255) to less than 50, greatly compressing the image processing time.

[0040] The aforementioned machine vision-based waste plastic pose recognition method, in step S4, specifically involves: extracting the contour of the binarized image after regional empirical threshold segmentation using the Suzuki algorithm. The Suzuki algorithm performs topological analysis on the digital binary image and creatively uses the outer boundary, hole boundary, and their hierarchical relationship to represent the contour of the digital image. The outer boundary and hole boundary are defined as follows: If there is a 1-connected region S1 and a 0-connected region S2, and S2 directly surrounds S1, then the boundary between S2 and S1 is called the outer boundary; if S1 directly surrounds S2, then the edge between S2 and S1 is called the hole boundary. Both the outer boundary and the hole boundary consist of 1 pixel. The surrounding of a connected region means that for two adjacent connected regions S1 and S2, if any point on S1 can reach S2 in any of the four directions, then S2 surrounds S1. The definition of the hierarchical relationship between contours is as follows: Assume there are 1-connected regions S1 and S3, and 0-connected region S2; S2 directly wraps around S1, S3 directly wraps around S2, the boundary between S1 and S2 is B1, and the boundary between S2 and S3 is B2. Then B2 is the parent boundary of B1. If S2 is the background, then the parent boundary of B1 is the image bounding box. The topological relationship between the image and the bounding box is as follows: Figure 6 As shown. The Suzuki algorithm extracts contours using a method similar to raster scanning, which involves scanning image pixels from left to right and top to bottom. By employing a boundary tracking algorithm, multiple contours of the image can be obtained, and each contour is assigned a unique number. During the scanning process, NBD represents the number of the currently tracked boundary, and LNBD represents the number of the previously saved boundary.

[0041] The specific process of Suzuki's algorithm for extracting the original contour is as follows: If the input image is F = {f(i, j)}, where i and j are the x and y coordinates of the pixel, respectively, the initial NBD is set to 1, the image border is regarded as the first contour boundary, and the image is scanned. When a pixel f(i, j) ≠ 0 is scanned, the following steps are performed:

[0042] (1) Determine which of the following situations?

[0043] (a) If f(i,j) = 1 and f(i,j-1) = 0, then (i,j) is the starting point of the outer boundary. Let NBD = NBD + 1, and at the same time save the coordinates of (i2,j2) as (i,j-1).

[0044] (b) If f(i,j)≥1 and f(i,j+1)=0, then (i,j) is the starting point of the hole boundary. Let NBD=NBD+1, and at the same time save the coordinates of (i2,j2) as (i,j+1).

[0045] (c) Other cases, skip to (4).

[0046] (2) Based on the saved previous boundary and the currently encountered boundary, query the parent boundary of the current boundary from Table 3, where B' is the previous boundary and B is the new boundary;

[0047] Table 3 Parent Boundary of the Current Boundary

[0048]

[0049] (3) Starting from the boundary starting point (i, j), perform boundary tracking according to the following algorithm.

[0050] (3.1) Taking (i, j) as the center and (i2, j2) as the starting point, search for the eight neighbors of (i, j) in a clockwise direction and determine whether there is a non-zero pixel. If a non-zero pixel is found, set the coordinates of the first non-zero pixel in the clockwise direction to (i1, j1) and go to (3.2); otherwise, set f(i, j) = -NBD and go to (4);

[0051] (3.2) Update the coordinates of (i2, j2) to (i1, j1) and the coordinates of (i3, j3) to (i, j);

[0052] (3.3) Centered on (i3, j3), starting from (i2, j2), search counterclockwise for the existence of non-zero pixels in the eight neighborhoods of (i3, j3), and let (i4, j4) be the coordinates of the first non-zero pixel encountered. The eight neighborhoods refer to the eight adjacent pixels around a pixel. That is, for a given pixel (i, j), the position of its eight neighborhoods can be represented as (i-1, j-1), (i-1, j), (i-1, j+1), (i, j-1), (i, j), (i, j+1), (i+1, j-1), (i+1, j+1), (i+1, j+1). In contrast, if only four neighborhoods (top, bottom, left, right) are considered, the boundary information in the diagonal direction may be ignored, resulting in incomplete or inaccurate extracted contours.

[0053] (3.4) If (i3, j3+1) is a pixel point checked in (3.3) and is a zero pixel point, then let f(i3, j3) = -NBD; if (i3, j3+1) is not a pixel point checked in (3.3) and is a non-zero pixel point, then let f(i3, j3) = NBD.

[0054] (3.5) If (i4, j4) = (i, j) and (i3, j3) = (i2, j2), it means that we have returned to the starting point and jump to (4); otherwise, update the coordinates of (i2, j2) to (i3, j3) and the coordinates of (i3, j3) to (i4, j4), and go to (3.3);

[0055] (4) If f(i,j)≠1, then LNBD=|f(i,j)|, continue scanning from point (i,j+1), and the algorithm ends when the bottom right corner vertex of the image is scanned.

[0056] The original Suzuki algorithm is improved and simplified. The Suzuki algorithm process used in this invention is as follows: Let f(i, j) represent the pixel value of point (i, j). Scan the binary image of waste plastic row-by-row until a point (m, n) in a connected region is encountered whose pixel value f(m-1, n) = 0 and f(m, n) = 255. This point is the first point on the contour. Starting from point (m-1, n), traverse the eight neighbors of point (m, n) counterclockwise. Starting from the first non-zero point found, traverse the eight neighbors of point (m, n) counterclockwise. The first non-zero point found is the next point on the contour. Continue this method until the contour is completely closed. Mark the pixels on the boundary of this connected region and assign a unique identifier to this boundary. After discovering a new connected region, increment the identifier by 1 and follow the same method to trace the contour. After obtaining all the contours of the waste plastic, select the largest one and output it.

[0057] After obtaining the contour of each region of the waste plastic, this invention uses the moments of the contour to determine the centroid position of each region of the waste plastic. The (p+q)th order contour moment m(pq) of the i-th region. i The definition is as follows:

[0058]

[0059] Where x and y are the horizontal and vertical coordinates of a pixel on the contour, I(x, y) is the pixel value (0 or 1) at pixel coordinate (x, y), and n i It is the number of pixels on the contour of the i-th region; when solving for the centroid, the first-order contour moment is used, that is, the values ​​of the coefficients p and q are 0 or 1 respectively, but the sum of the two is 1, that is, p + q = 1.

[0060] In the contour image, all pixel values ​​of points on the contour are 1, therefore the zeroth moment m(00) is... i It is the number of points on the contour of the i-th region, and the first-order contour moment m(10). i m(01) i These are the summations of the x and y coordinates of each pixel on the contour of the i-th region, respectively. The centroid position (x, y) corresponding to the contour of the i-th region is also shown. i y i ) can be calculated using equation (3).

[0061]

[0062] The contours of the binarized image were extracted and the centroid positions were determined using the method described above. The results are as follows: Figure 7 As shown. After calculating the centroid coordinates of each region, the following formulas are used to calculate the total centroid (x, y) of the waste plastic and the angle θ of the waste plastic.

[0063]

[0064]

[0065] Where n is the number of connected components, (x i y i ) is the centroid of the contour of the i-th connected domain.

[0066] Using the above formula to Figure 7 The centroid, angle, and calculation time (N=12) of the waste plastic bottles are shown in Table 4 below.

[0067] Table 4. Calculation results of the pose of waste plastics

[0068]

[0069] As can be seen from the table above, the empirical threshold segmentation method using regional division can effectively reduce the computational load of threshold segmentation, reducing the computation time to 23ms, which greatly reduces the time consumed by image recognition.

[0070] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for recognizing the pose of waste plastics based on machine vision, characterized in that, Includes the following steps: Step S1: Install the industrial color camera above the conveyor belt. When waste plastic appears within the field of view of the industrial color camera, the industrial color camera will capture images of the waste plastic in real time. Step S2: Use a weighted average method to convert the image obtained in step S1 to grayscale to distinguish the waste plastic from the background in the original image; Step S3: The grayscale image obtained in step S2 is segmented into multiple horizontal regions corresponding to the camera's field of view using the horizontal average cutting method. Then, a fixed range of thresholds is set for each region to perform empirical threshold segmentation. After threshold segmentation, a binarized image is obtained. Step S4: Use the Suzuki algorithm to extract contours from multiple regions of the binarized image obtained by threshold segmentation in step S3. Then, use contour moments to calculate the position of the centroid of multiple regions. Finally, calculate the centroid and angle of the waste plastic as a whole based on the position of the centroid of each region, and finally realize the pose recognition of waste plastic. The formulas for calculating the center of mass and angle of the entire waste plastic are as follows: Where (x, y) are the centroid coordinates of the entire waste plastic system, and n is the number of regions, (x i y i ) is the centroid of the contour in the i-th region; θ is the angle of the entire waste plastic; The formula for calculating the centroid of the contour in the i-th region is: Where, m(00) i It is the number of points on the contour of the i-th region, m(10). i m(01) i These are the summation of the x and y coordinates of each pixel on the contour of the i-th region.

2. The method for recognizing the pose of waste plastics based on machine vision according to claim 1, characterized in that, The specific method of weighted average grayscale processing in step S2 is as follows: the image acquired by the industrial color camera is multiplied by different weights according to the importance of the three different color channels of RGB, and then the weights are added together to obtain a grayscale value, thus converting the RGB image into a grayscale image.

3. The method for recognizing the pose of waste plastics based on machine vision according to claim 1, characterized in that, The specific method for the horizontal average cutting and empirical threshold segmentation in step S3 is as follows: First, define the cutting rows, calculate the height of each region according to the height of the image, then cut the image horizontally into several regions according to the predefined number of rows, and for each region, calculate the average gray value of the region with only the conveyor belt background, then set the threshold traversal range to distinguish the background and foreground according to the average gray value, and finally perform separate threshold segmentation on each region to generate the corresponding binarized image.

4. The method for recognizing the pose of waste plastics based on machine vision according to claim 1, characterized in that, The specific method for step S4, "Suzuki algorithm for contour extraction," is as follows: Initialize the binary image with a marker image of the same size, and initialize all pixels to 0. Let f(i,j) represent the pixel value of point (i,j). Scan the binary image of waste plastic row by row until a point (m,n) in a connected region is encountered, where f(m-1,n) = 0 and f(m,n) = 255. This point is the first point on the contour. Using point (m-1,n) as the starting point, traverse the eight neighbors of point (m,n) counterclockwise. Starting from the first non-zero point found, traverse the eight neighbors of point (m,n) counterclockwise. The first non-zero point found is the next point on the contour. Continue this process until the contour is completely closed. Mark the pixels on the boundary of this connected region and assign a unique identifier to this boundary. After discovering a new connected region, increment the identifier by 1 and track the contour in the same way. After obtaining all the contours of the waste plastic, select the largest one and output it to obtain the complete contour image.

Images