Recyclable garbage recognition method based on input channel improved BlendMask

Abstract

The application discloses a recyclable garbage recognition method based on input channel improved BlendMask, relates to the garbage recycling technical field, and aims at the problem that the prior art cannot accurately obtain the quality of various recyclable garbage in garbage. The application firstly utilizes a single garbage cutout and splicing simulation mixed garbage image obtained through crawling to make up for the vacancy of a domestic garbage image dataset available for image segmentation, improves the input channel on the basis of an existing image segmentation model, and obtains a model which can obtain the proportion of various recyclable materials in domestic garbage by only uploading a picture taken by a camera, can accurately obtain the content of various recyclable garbage, and can further obtain the quality of various classifications of recyclable garbage according to the total quality of a garbage pile to be recycled.

Description

Technical Field

[0001] This invention relates to the field of waste recycling technology, specifically to a recyclable waste identification method based on an improved BlendMask input channel. Background Technology

[0002] The volume of urban household waste collected in China increased from 148.565 million tons in 2003 to 242.062 million tons in 2019. This dramatic increase has led to a series of environmental, economic, and social problems. Recyclable waste, including cardboard, glass, plastic, and metal, is subject to incineration or landfill disposal, which represents a serious waste of resources. Sorting and classifying household waste to identify recyclable components can effectively reuse resources from waste, reduce the amount of waste entering incineration and landfill facilities, and mitigate the threat posed by household waste. Analyzing household waste in a simple and quick way before sorting to obtain the content of various recyclable materials facilitates subsequent sorting and disposal. Current technologies typically rely on manual observation, but this method cannot accurately and efficiently obtain the quality of various recyclable materials in multiple waste piles within a given area. Summary of the Invention

[0003] The purpose of this invention is to address the problem that existing technologies cannot accurately obtain the quality of various recyclable wastes in garbage, and to propose a recyclable waste identification method based on an improved BlendMask input channel.

[0004] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0005] An improved BlendMask method for identifying recyclable waste based on input channels, the method comprising the following steps:

[0006] Step 1: Obtain labeled images containing only one type of household waste, and then perform data cleaning on the household waste images using OpenCV;

[0007] Step 2: Extract the single type of waste marked in the cleaned household waste image to obtain a single waste image. Then, stitch the single waste images together to obtain a mixed waste image. Adjust the scaling ratio to obtain an adjusted mixed waste image. The size of the waste in the adjusted mixed waste image is consistent with the actual proportions.

[0008] Step 3: Classify and label the adjusted mixed waste images according to the waste categories to obtain dataset 1 labeled with waste categories. The waste categories include dry waste, wet waste, hazardous waste and recyclable waste. Then, label the recyclable waste in dataset 1 according to the detailed classification of recyclable waste to obtain dataset 2 containing multiple detailed classification solid waste labels.

[0009] Step 4: Perform data augmentation on dataset 1 and dataset 2 respectively;

[0010] Step 5: Feed the augmented dataset 1 and dataset 2 into the improved BlendMask model for training. Then, adjust the hyperparameters and obtain the optimal hyperparameter settings based on the cross-validation method. This will result in image segmentation model 1 for identifying major waste categories and image segmentation model 2 for identifying detailed recyclable waste categories.

[0011] The improved BlendMask model has five channels: RGB primary color channels, grayscale channels, and contour channels.

[0012] The improved BlendMask model includes a Detector module and a BlendMask module. The Detector module is used to perform global segmentation and boundary detection on the input features, and the BlendMask module is used to perform local segmentation and channel information feature fusion.

[0013] The improved BlendMask model also introduces a channel attention mechanism based on SE Block. This channel attention mechanism includes a Squeeze module and an Excitation module. The Squeeze module is used to compress the input image for each channel and then perform global average pooling on the compressed image. The Excitation module uses an activation algorithm to calculate the importance of each channel and finally uses a re-weighting operation to use the importance of each channel as the calculation weight of the BlendMask model.

[0014] Step 6: The waste pile to be recycled is tumbled and stirred multiple times. After each stirring, the surface image of the waste pile is acquired and input into image segmentation model 1 to obtain the output mask segmentation result set 1.

[0015] Step 7: Input the mask segmentation result set 1 into the image segmentation model 2 to obtain the mask segmentation result set 2;

[0016] Step 8: Identify the pixels in the mask segmentation result set 2 to obtain the identification results of each detailed category of recyclable waste. Then, sum and average the identification results of multiple tumbling and stirring to obtain the average area ratio of each detailed category of recyclable waste in the total waste, as well as the average surface area ratio of each subcategory of recyclable waste.

[0017] Step Nine: Repeat Steps Six through Eight to process different piles within the same area, obtaining the surface area ratio of different types of recyclable waste. Then, based on the spatial geographical location of each pile, perform processing using an improved Kriging interpolation algorithm to obtain Kriging weights. Finally, based on these Kriging weights, obtain the spatial distribution of the proportion of each subcategory of recyclable waste. The improved Kriging interpolation algorithm processing is expressed as follows:

[0018] r 11 a1+r 12 a2+…+r 1n a n +φ=r 10

[0019] r 21 a1+r 22 a2+…+r 2n a n +φ=r 20

[0020] …

[0021] r n1 a1+r n2 a2+…+r nn a n +φ=r n0

[0022]

[0023] Where, r n0 r is the difference between the actual percentage of recyclable waste in the nth pile and the average percentage. nn The semivariance function is calculated based on the spatial correlation between sampling location pairs, representing the proportion of recyclable waste between each pair of piles. Φ is the error term, and a j The spatial weights of 1 to n heap volumes;

[0024] The spatial distribution of the proportion of each subcategory of recyclable waste is represented as follows:

[0025]

[0026] in, For unknown point data values, z iLet be the proportion of recyclable waste in the i-th waste pile, and c be the expected value obtained by a simple averaging method of the data at unknown points;

[0027] Step 10: Obtain the total mass of different waste piles. Then, based on the spatial distribution of the proportion of each subcategory of recyclable waste, multiply the average area proportion of each subcategory of recyclable waste in different piles in the spatial distribution by the corresponding total mass of the pile to obtain the mass of each subcategory of recyclable waste in different waste piles within the region.

[0028] Furthermore, the data cleaning includes filtering image noise removal, size uniform adjustment, rotation and flipping to enhance the viewing angle, and cropping irrelevant edge areas.

[0029] Furthermore, in step four, data augmentation is performed using the Cutout method and style transfer transformation method. The data augmentation includes brightness and contrast adjustment, edge smoothing, and color balance adjustment.

[0030] Furthermore, the hyperparameters include the resolution R of the bottom-level RoI of the BlendMask module in the BlendMask model, the resolution M of the top-level prediction (A), the input features of the bottom module, the sampling method of the bottombases, and the interpolation method of the top-level attention.

[0031] Furthermore, the images of household waste include images of recyclable waste, wet waste, hazardous waste, and dry waste. The images of recyclable waste include images of cardboard boxes, plastic bottles, glasses, milk cartons, aluminum cans, metal cans, clothes, pants, shoes, plush toys, hats, bags, toys, books, newspapers, paper bags, power strips, circuit boards, and envelopes.

[0032] Furthermore, the recyclable waste includes cardboard boxes, plastic bottles, glasses, milk cartons, aluminum cans, metal cans, clothes, pants, shoes, plush toys, hats, bags, toys, books, newspapers, paper bags, power strips, and circuit boards.

[0033] Furthermore, in step two, the obtained single garbage images are stitched together using the Python extension library Pillow.

[0034] Furthermore, the labeling in step three is performed using the EISeg annotation software under the Paddle framework.

[0035] Furthermore, in step one, the acquisition of labeled images containing only a single type of household waste is performed using web crawling technology. The specific steps are as follows:

[0036] Web crawlers were used to obtain images containing only a single type of household waste. Damaged images were then identified, and the minHash algorithm was used to remove duplicates. Finally, the Laplacian operator was used to remove blurry images from the deduplicated images.

[0037] Furthermore, in step two, the scaling ratio is adjusted using an iterative algorithm.

[0038] The beneficial effects of this invention are:

[0039] This application is the first to utilize crawled single garbage images, which are then stitched together to simulate mixed garbage images, filling the gap in the availability of household waste image datasets for image segmentation. Based on existing image segmentation models, the input channel is improved to obtain a model that can obtain the proportion of various recyclable materials in household waste simply by taking and uploading an image. It can accurately obtain the content of various recyclable wastes, and based on the overall mass of the garbage pile to be recycled, the mass of each category of recyclable waste can be obtained.

[0040] It serves the sorting and classification of household waste, promotes the resource utilization of recyclable waste, and has broad application prospects in the field of intelligent solid waste management in multiple scenarios. Attached Figure Description

[0041] Figure 1 This is the overall flowchart of this application;

[0042] Figure 2 This is a schematic diagram of a single type of household waste.

[0043] Figure 3 This is a schematic diagram of a composite image of mixed household waste.

[0044] Figure 4 This is a schematic diagram of a recyclable waste labeling dataset;

[0045] Figure 5 A schematic diagram of the entire garbage annotation dataset;

[0046] Figure 6 This is a schematic diagram of the mask segmentation results of a domestic waste image output by two models. Detailed Implementation

[0047] It should be noted that, where there is no conflict, the various embodiments disclosed in this application can be combined with each other.

[0048] Specific implementation method one: Refer to Figure 1This embodiment describes a recyclable waste identification method based on an improved BlendMask input channel, which includes the following steps:

[0049] Step 1: Obtain labeled images containing only one type of household waste, and then perform data cleaning on the household waste images using OpenCV;

[0050] Step 2: Extract the single type of waste marked in the cleaned household waste image to obtain a single waste image. Then, stitch the single waste images together to obtain a mixed waste image. Adjust the scaling ratio to obtain an adjusted mixed waste image. The size of the waste in the adjusted mixed waste image is consistent with the actual proportions.

[0051] Step 3: Classify and label the adjusted mixed waste images according to the waste categories to obtain dataset 1 labeled with waste categories. The waste categories include dry waste, wet waste, hazardous waste and recyclable waste. Then, label the recyclable waste in dataset 1 according to the detailed classification of recyclable waste to obtain dataset 2 containing multiple detailed classification solid waste labels.

[0052] Step 4: Perform data augmentation on dataset 1 and dataset 2 respectively;

[0053] Step 5: Feed the augmented dataset 1 and dataset 2 into the improved BlendMask model for training. Then, adjust the hyperparameters and obtain the optimal hyperparameter settings based on the cross-validation method. This will result in image segmentation model 1 for identifying major waste categories and image segmentation model 2 for identifying detailed recyclable waste categories.

[0054] The improved BlendMask model has five channels: RGB primary color channels, grayscale channels, and contour channels.

[0055] The improved BlendMask model includes a Detector module and a BlendMask module. The Detector module is used to perform global segmentation and boundary detection on the input features, and the BlendMask module is used to perform local segmentation and channel information feature fusion.

[0056] The improved BlendMask model also introduces a channel attention mechanism based on SE Block. This channel attention mechanism includes a Squeeze module and an Excitation module. The Squeeze module is used to compress the input image for each channel and then perform global average pooling on the compressed image. The Excitation module uses an activation algorithm to calculate the importance of each channel and finally uses a re-weighting operation to use the importance of each channel as the calculation weight of the BlendMask model.

[0057] Step 6: The waste pile to be recycled is tumbled and stirred multiple times. After each stirring, the surface image of the waste pile is acquired and input into image segmentation model 1 to obtain the output mask segmentation result set 1.

[0058] Step 7: Input the mask segmentation result set 1 into the image segmentation model 2 to obtain the mask segmentation result set 2;

[0059] Step 8: Identify the pixels in the mask segmentation result set 2 to obtain the identification results of each detailed category of recyclable waste. Then, sum and average the identification results of multiple tumbling and stirring to obtain the average area ratio of each detailed category of recyclable waste in the total waste, as well as the average surface area ratio of each subcategory of recyclable waste.

[0060] Step Nine: Repeat Steps Six through Eight to process different piles within the same area, obtaining the surface area ratio of different types of recyclable waste. Then, based on the spatial geographical location of each pile, perform processing using an improved Kriging interpolation algorithm to obtain Kriging weights. Finally, based on these Kriging weights, obtain the spatial distribution of the proportion of each subcategory of recyclable waste. The improved Kriging interpolation algorithm processing is expressed as follows:

[0061] r 11 a1+r 12 a2+…+r 1n a n +φ=r 10

[0062] r 21 a1+r 22 a2+…+r 2n a n +φ=r 20

[0063] …

[0064] r n1 a1+r n2 a2+…+r nn a n +φ=rn0

[0065]

[0066] Where, r n0 r is the difference between the actual percentage of recyclable waste in the nth pile and the average percentage. nn The semivariance function is calculated based on the spatial correlation between sampling location pairs, representing the proportion of recyclable waste between each pair of piles. Φ is the error term, and a j The spatial weights of 1 to n heap volumes;

[0067] The spatial distribution of the proportion of each subcategory of recyclable waste is represented as follows:

[0068]

[0069] in, For unknown point data values, z i Let be the proportion of recyclable waste in the i-th waste pile, and c be the expected value obtained by a simple averaging method of the data at unknown points;

[0070] Step 10: Obtain the total mass of different waste piles. Then, based on the spatial distribution of the proportion of each subcategory of recyclable waste, multiply the average area proportion of each subcategory of recyclable waste in different piles in the spatial distribution by the corresponding total mass of the pile to obtain the mass of each subcategory of recyclable waste in different waste piles within the region.

[0071] The improved BlendMask model comprises a Detector module and a BlendMask module. The detector module acts as a detector for input channel feature information, performing global segmentation and boundary detection on the input features. The BlendMask module performs local segmentation and channel feature fusion. The two modules combine information fusion to improve the accuracy of image segmentation and recognition. Compared to the traditional BlendMask, the improved BlendMask model of this invention increases the number of input channels. This invention uses five channels: RGB primary color channels, grayscale channels, and contour channels, enhancing its ability to recognize solid waste images. Further improvements to the model include the introduction of a channel attention mechanism based on SE Block to address the issues of the basic BlendMask model's single channel and unbalanced weight distribution. The SE Block module of this invention enables lightweight adaptive adjustment of different channel weights, reducing the computational overhead of the original network and allowing for better accurate identification of solid waste types. This results in a BlendMask model with improved recognition performance and efficiency.

[0072] Based on the improved Kriging interpolation algorithm, it is assumed that the distribution of recyclable household waste in space exhibits isotropic Kriging characteristics. Using the Lagrange multiplier method, the following equation can be constructed to solve the Kriging problem of the distribution of recyclable waste proportions among different types of waste: The core formula of the algorithm is as follows:

[0073]

[0074]

[0075] This formula represents the numerical relationship between n spatial points. C(si, sj) is the covariance coefficient between any two points, which is derived based on the Euclidean geometric distance between the two points. It is assumed that the Kriging distribution of each type of recyclable waste contains n samples from n piles, and a1, a2…an correspond to n weight coefficients, the sum of which is one. f(a1, a2…an, λ) is the Kriging interpolation for the unknown point. This method is repeatedly applied to all types of waste, inputting this point, to determine the approximate proportion of different types of recyclable waste at different points in the space. For a single type of recyclable waste, the above equations can be extended to n+1 equations, enabling the solution of the Kriging spatial weights at any point, i.e.:

[0076] r 11 a1+r 12 a2+…+r 1n a n +φ=r 10

[0077] r 21 a1+r 22 a2+…+r 2n a n +φ=r 20

[0078] …

[0079] r n1 a1+r n2 a2+…+r nn a n +φ=r n0

[0080]

[0081] By inputting the percentage value r of this type of recyclable waste in the analyzed pile, the weighting coefficient can be solved, the spatial correlation of the recyclable waste percentage distribution between different piles can be clarified, the Kriging weight for a certain type of recyclable waste at any spatial point can be obtained, and the spatial distribution of the percentage can be obtained.

[0082] Where, r n0r is the difference between the actual percentage of recyclable waste in the nth pile and the average percentage. nm The semivariance function representing the proportion of recyclable waste between the nth and mth piles is calculated based on the spatial correlation between sampling location pairs, where Φ is the error term and a j The range is 1-n, a j Let each of the 1-n piles have its own spatial weight. This system of equations can be used to solve for the spatial weights of different piles. The purpose of this system of equations is to ensure that the sum of the spatial weights is unique, thereby optimizing the spatial distribution of the unknown waste proportion data. After obtaining the spatial weights of different piles, the waste classification proportion at unknown locations is obtained based on the following interpolation formula:

[0083] For unknown point data values, z i Let represent the proportion of recyclable waste in the i-th waste pile, and c be the expected value obtained by a simple averaging method for the data at unknown points.

[0084] The SE Block-based channel attention mechanism includes a Squeeze module and an Excitation module. The Squeeze module compresses the input images for each channel and performs global average pooling on the compressed images to obtain a one-dimensional parameter-represented receptive field for the five-channel image. The Excitation module uses an activation algorithm to calculate the importance of each channel and finally uses a re-weighting operation to use the importance of each channel as the calculation weight of the BlendMask model. Compared with the existing SE Block model, the model of this invention introduces a multi-scale integration mechanism of image semantic context information, realizing joint perception of global and local image semantic context information based on pixel grouping. By encoding the information of different channels as a whole, the objectivity and accuracy of weighting in the recognition process of different channels are improved.

[0085] The SEblock module in this application implements lightweight adaptive adjustment of different channel weights, reducing the computational overhead of the original network and enabling the network to better and more accurately identify solid waste types, thereby obtaining a BlendMask model with improved identification performance and efficiency.

[0086] Specific Implementation Method Two: This implementation method is a further explanation of Specific Implementation Method One. The difference between this implementation method and Specific Implementation Method One is that the data cleaning includes filtering image noise removal, size uniform adjustment, rotation and flipping to enhance the viewing angle, and cropping of irrelevant edge areas.

[0087] Specific Implementation Method 3: This implementation method is a further explanation of Specific Implementation Method 1. The difference between this implementation method and Specific Implementation Method 1 is that the data enhancement in step four is performed through the Cutout method and the style transfer change method. The data enhancement includes brightness and contrast adjustment, edge smoothing, and color balance adjustment.

[0088] Specific Implementation Method Four: This implementation method is a further explanation of Specific Implementation Method One. The difference between this implementation method and Specific Implementation Method One is that the hyperparameters include the resolution R of the bottom-level RoI of the BlendMask module in the BlendMask model, the resolution M of the top-level prediction (A), the input features of the bottom module, the sampling method of the bottom bases, and the interpolation method of the top-level attention.

[0089] Specific Implementation Method Five: This implementation method is a further explanation of Specific Implementation Method One. The difference between this implementation method and Specific Implementation Method One is that the images of household waste include images of recyclable waste, wet waste, hazardous waste, and dry waste. Images of recyclable waste include images of cardboard boxes, plastic bottles, glasses, milk cartons, aluminum cans, metal cans, clothes, pants, shoes, plush toys, hats, bags, toys, books, newspapers, paper bags, power strips, circuit boards, and envelopes.

[0090] Specific Implementation Method Six: This implementation method is a further explanation of Specific Implementation Method One. The difference between this implementation method and Specific Implementation Method One is that the recyclable waste includes cardboard boxes, plastic bottles, glass cups, glass bottles, milk cartons, aluminum cans, metal cans, clothes, pants, shoes, plush toys, hats, bags, toys, books, newspapers, paper bags, power strips, and circuit boards.

[0091] Specific Implementation Method Seven: This implementation method is a further explanation of Specific Implementation Method One. The difference between this implementation method and Specific Implementation Method One is that in step two, the single garbage images obtained are stitched together using the Python extension library Pillow.

[0092] Specific Implementation Method Eight: This implementation method is a further explanation of Specific Implementation Method One. The difference between this implementation method and Specific Implementation Method One is that the annotation in step three is performed using the EISeg annotation software under the Paddle framework.

[0093] Specific Implementation Method Nine: This implementation method is a further explanation of Specific Implementation Method One. The difference between this implementation method and Specific Implementation Method One is that in step one, obtaining labeled images containing only a single type of household waste is done using web crawling technology. The specific steps are as follows:

[0094] Web crawlers were used to obtain images containing only a single type of household waste. Damaged images were then identified, and the minHash algorithm was used to remove duplicates. Finally, the Laplacian operator was used to remove blurry images from the deduplicated images.

[0095] Specific Implementation Method Ten: This implementation method is a further explanation of Specific Implementation Method One. The difference between this implementation method and Specific Implementation Method One is that the scaling ratio is adjusted in step two by an iterative algorithm.

[0096] This application addresses the lack of methods for analyzing the recyclable content in municipal solid waste. It innovatively proposes a method that synthesizes mixed waste images from single waste images and trains an improved deep learning model on these images. This results in a method that can obtain the proportion of various recyclable waste types within municipal solid waste from images. The aim is to directly analyze municipal solid waste image information using artificial intelligence methods, quickly and easily obtaining the proportion of each type of recyclable waste. After training, this model can be applied to real waste piles; by inputting a waste image, it can output the proportion of each type of recyclable material in the waste pile, facilitating the resource utilization of recyclable waste and the implementation of sorting and disposal processes.

[0097] This application first uses web crawling technology to acquire images of household waste. After segmenting the images into training, validation, and test groups, it randomly combines single types of waste to simulate mixed waste through image cutout and stitching. Two datasets are obtained through two labeling processes: one for recyclable waste and one for all waste. Data augmentation is then performed to expand the datasets after labeling. Finally, an improved BlendMask model is constructed, increasing the input from three channels of a color image to RGB + grayscale + outline. Figure 5 The system uses channels to allow more features to be input into the model simultaneously. Two datasets are fed into the models for training, and hyperparameters are adjusted to achieve optimal results, resulting in Model 1 for predicting recyclable waste and Model 2 for predicting all waste. After the predicted images are fed into the two models, pixel-by-pixel statistics are performed on the mask segmentation results output by both models to obtain the proportion of each type of recyclable waste in the total waste for a single image. A weighted average of the results from multiple images represents the volume ratio of each type of recyclable material in this portion of household waste. The specific steps are as follows:

[0098] Step 1: Use web crawler technology to obtain images of household waste and perform data cleaning. The images are divided into recyclable waste, wet waste, hazardous waste, and dry waste. Recyclable waste is further subdivided into 20 categories: cardboard boxes, plastic bottles, glass cups, glass bottles, milk cartons, aluminum cans, metal cans, clothes, pants, shoes, plush toys, hats, bags, toys, books, newspapers, paper bags, power strips, circuit boards, and envelopes.

[0099] Step 2 divides the acquired images into 3 groups in an 8:1:1 ratio, which are used to construct the training set, validation set, and test set, respectively.

[0100] Step 3 involves cutting out individual pieces of trash from each of the three sets of images. Then, using the Python extension library Pillow, these individual trash images are randomly combined and stitched together to simulate mixed trash. Each image contains 1 to 10 individual pieces of trash, allowing negative samples without recyclable trash. The training, validation, and test sets are in an 8:1:1 ratio. During the stitching process, an appropriate scaling ratio is selected based on the image size and the actual size of the objects to ensure that the size proportions between different objects are close to those of real objects.

[0101] Step 4: Use software to perform image segmentation and annotation on the mixed garbage images. Each image is annotated twice: once only the recyclable garbage in the image is annotated to obtain dataset 1, and once all the garbage is annotated to obtain dataset 2.

[0102] Step 5 uses color dithering, random grayscale conversion, and random lighting transformation techniques to augment the data and expand the dataset.

[0103] Step 6: Construct an improved BlendMask model, changing the input from a three-channel color image to RGB + grayscale + outline. Figure 5 Channels allow more features to be fed into the model simultaneously, improving its performance in identifying recyclable waste. BlendMask consists of a one-stage object detection network FCOS and a mask branch. The mask branch has three parts: a bottom module processes the low-level features to generate a score map (Base); a top layer is strung on the detector head to generate the top-level attention mechanism (Attention) corresponding to the Base; and finally, a blending module (Blender) fuses the Base and Attention.

[0104] Step 7: Feed the training sets of Dataset 1 and Dataset 2 into the model for training. Adjust the hyperparameters based on the prediction results on the validation set to obtain Model 1, which predicts recyclable waste, and Model 2, which predicts all waste. Run the model on the test set to obtain the model's accuracy.

[0105] Step 8 involves tumbling the waste pile requiring recycling potential assessment 10 times. After each tumbling, the surface of the waste pile is photographed. These surface images are input into Model 1 and Model 2. The output mask segmentation results represent recyclable waste and all waste, respectively. The pixel counts of these two models are statistically analyzed, and the ratio of the number of pixels represents the proportion of each type of recyclable waste to the total waste. Using the pixel count of all waste in the image as a weight, the prediction results from 10 images are weighted and averaged to obtain the proportion of each type of recyclable material in this portion of municipal solid waste. Finally, the mask segmentation results are output as a PNG image, and the analysis report is output as a CSV file.

[0106] Example:

[0107] (1) Approximately 20,000 images of recyclable waste, wet waste, hazardous waste, and dry waste were obtained using web crawling technology. Recyclable waste was further subdivided into 20 categories: cardboard boxes, plastic bottles, glass cups, milk cartons, aluminum cans, metal cans, clothes, pants, shoes, plush toys, hats, bags, toys, books, newspapers, paper bags, power strips, circuit boards, and envelopes. The data is as follows: Figure 2 As shown, the acquired images were divided into training, validation, and test groups in an 8:1:1 ratio.

[0108] (2) The three sets of images were cut out separately, and then the individual pieces of trash were randomly combined and stitched together using the Python extension library Pillow. The individual pieces of trash were selected randomly each time, so some images were reused multiple times, resulting in approximately 20,000 mixed trash images. The stitched images are shown below. Figure 3 As shown.

[0109] (3) The EISeg annotation software under the Paddle framework was used to annotate the mixed household waste images. Each image was annotated twice: once for recyclable waste and once for all waste. Data augmentation was performed on each image. The resulting two datasets are shown below. Figure 4 , Figure 5 As shown.

[0110] (4) The grayscale and contour maps of the image were obtained using OpenCV. The contour map was obtained by calculating the image gradient using the Scharr operator. All five channels were then simultaneously fed into the BlendMask model. The model was trained using the training set, and the hyperparameters were adjusted based on the validation set. A total of 10,000 training iterations were performed. The resolution of the bottom-level RoI was set to 56, the resolution of the top-level prediction (A) was set to 7, and the number of bases was set to 4. The model performed well on the test set.

[0111] (5) Scrape a small number of images to create 10 mixed garbage images, simulate the household waste to be predicted, input them into the model, and the mask output results are as follows: Figure 6 As shown in Table 1, the percentage of each type of recyclable waste is as follows.

[0112] Table 1. Analysis Results of the Proportion of Various Types of Recyclable Waste

[0113]

[0114] It should be noted that the specific embodiments are merely explanations and illustrations of the technical solution of the present invention and should not be used to limit the scope of protection. Any modifications made in accordance with the claims and specification of the present invention that are only partial should still fall within the protection scope of the present invention.

Claims

1. An improved BlendMask method for identifying recyclable waste based on input channels, characterized in that... The method includes the following steps: Step 1: Obtain labeled images containing only one type of household waste, and then perform data cleaning on the household waste images using OpenCV; Step 2: Extract the single type of waste marked in the cleaned household waste image to obtain a single waste image. Then, stitch the single waste images together to obtain a mixed waste image. Adjust the scaling ratio to obtain an adjusted mixed waste image. The size of the waste in the adjusted mixed waste image is consistent with the actual proportions. Step 3: Classify and label the adjusted mixed waste images according to the waste categories to obtain dataset 1 labeled with waste categories. The waste categories include dry waste, wet waste, hazardous waste and recyclable waste. Then, label the recyclable waste in dataset 1 according to the detailed classification of recyclable waste to obtain dataset 2 containing multiple detailed classification solid waste labels. Step 4: Perform data augmentation on dataset 1 and dataset 2 respectively; Step 5: Feed the augmented dataset 1 and dataset 2 into the improved BlendMask model for training. Then, adjust the hyperparameters and obtain the optimal hyperparameter settings based on the cross-validation method. This will result in image segmentation model 1 for identifying major waste categories and image segmentation model 2 for identifying detailed recyclable waste categories. The improved BlendMask model has five channels: RGB primary color channels, grayscale channels, and contour channels. The improved BlendMask model includes a Detector module and a BlendMask module. The Detector module is used to perform global segmentation and boundary detection on the input features, and the BlendMask module is used to perform local segmentation and channel information feature fusion. The improved BlendMask model also introduces a channel attention mechanism based on SE Block. The channel attention mechanism based on SE Block includes a Squeeze module and an Excitation module. The Squeeze module is used to compress the input image of each channel and then perform global average pooling on the compressed image. The Excitation module uses an activation algorithm to calculate the importance of each channel and finally uses a re-weighting operation to use the importance of each channel as the calculation weight of the BlendMask model. Step 6: The waste pile to be recycled is tumbled and stirred multiple times. After each stirring, the surface image of the waste pile is acquired and input into image segmentation model 1 to obtain the output mask segmentation result set 1. Step 7: Input the mask segmentation result set 1 into the image segmentation model 2 to obtain the mask segmentation result set 2; Step 8: Identify the pixels in the mask segmentation result set 2 to obtain the identification results of each detailed category of recyclable waste. Then, sum and average the identification results of multiple tumbling and stirring to obtain the average area ratio of each detailed category of recyclable waste in the total waste, as well as the average surface area ratio of each subcategory of recyclable waste. Step Nine: Repeat Steps Six through Eight to process different piles within the same area, obtaining the surface area ratio of different types of recyclable waste. Then, based on the spatial geographical location of each pile, perform processing using an improved Kriging interpolation algorithm to obtain Kriging weights. Finally, based on these Kriging weights, obtain the spatial distribution of the proportion of each subcategory of recyclable waste. The improved Kriging interpolation algorithm processing is expressed as follows: r 11 a1+r 12 a2+…+r 1n a n +φ=r 10 r 21 a1+r 22 a2+…+r 2n a n +φ=r 20 … r n1 a1+r n2 a2+…+r nn a n +φ=r n0 Where, r n0 r is the difference between the actual percentage of recyclable waste in the nth pile and the average percentage. nn The semivariance function is calculated based on the spatial correlation between sampling location pairs, representing the proportion of recyclable waste between each pair of piles. Φ is the error term, and a j The spatial weights of 1 to n heap volumes; The spatial distribution of the proportion of each subcategory of recyclable waste is represented as follows: in, For unknown point data values, z i Let be the proportion of recyclable waste in the i-th waste pile, and c be the expected value obtained by a simple averaging method of the data at unknown points; Step 10: Obtain the total mass of different waste piles. Then, based on the spatial distribution of the proportion of each subcategory of recyclable waste, multiply the average area proportion of each subcategory of recyclable waste in different piles in the spatial distribution by the corresponding total mass of the pile to obtain the mass of each subcategory of recyclable waste in different waste piles within the region.

2. The recyclable waste identification method based on the improved BlendMask input channel according to claim 1, characterized in that... The data cleaning includes filtering image noise removal, size uniform adjustment, rotation and flipping to enhance the viewing angle, and cropping irrelevant edge areas.

3. The recyclable waste identification method based on the improved BlendMask input channel according to claim 2, characterized in that... In step four, data augmentation is performed using the Cutout method and style transfer transformation method. The data augmentation includes brightness and contrast adjustment, edge smoothing, and color balance adjustment.

4. The recyclable waste identification method based on the improved BlendMask input channel according to claim 3, characterized in that... The hyperparameters include the resolution R of the bottom-level RoI of the BlendMask module in the BlendMask model, the resolution M of the top-level prediction (A), the input features of the bottom module, the sampling method of the bottombases, and the interpolation method of the top-level attention.

5. The recyclable waste identification method based on the improved BlendMask input channel according to claim 4, characterized in that... The images of household waste include images of recyclable waste, wet waste, hazardous waste, and dry waste. The images of recyclable waste include images of cardboard boxes, plastic bottles, glasses, milk cartons, aluminum cans, metal cans, clothes, pants, shoes, plush toys, hats, bags, toys, books, newspapers, paper bags, power strips, circuit boards, and envelopes.

6. The recyclable waste identification method based on the improved BlendMask input channel according to claim 5, characterized in that... The recyclable waste includes cardboard boxes, plastic bottles, glass cups, glass bottles, milk cartons, aluminum cans, metal cans, clothes, pants, shoes, plush toys, hats, bags, toys, books, newspapers, paper bags, power strips, and circuit boards.

7. The recyclable waste identification method based on the improved BlendMask input channel according to claim 1, characterized in that... In step two, the obtained single garbage images are stitched together using the Python extension library Pillow.

8. The recyclable waste identification method based on the improved BlendMask input channel according to claim 1, characterized in that... In step three, the labeling is performed using the EISeg annotation software under the Paddle framework.

9. The recyclable waste identification method based on the improved BlendMask input channel according to claim 1, characterized in that... In step one, obtaining labeled images containing only one type of household waste is performed using web crawling technology. The specific steps are as follows: Web crawlers were used to obtain images containing only a single type of household waste. Damaged images were then identified, and the minHash algorithm was used to remove duplicates. Finally, the Laplacian operator was used to remove blurry images from the deduplicated images.

10. The recyclable waste identification method based on the improved BlendMask input channel according to claim 1, characterized in that... In step two, the scaling ratio is adjusted using an iterative algorithm.

Images