A waste and miscellaneous plastic detection method based on improved M2Det

Abstract

The application discloses a kind of based on the waste plastic detection method of improvement M2Det, the described method includes: through camera real-time sampling and the way of web crawler, collection multiple waste plastic data;Original picture is randomly divided into training set, verification set and test set;Through data enhancement method expansion existing data set and obtain a large amount of data;Using improved M2Det algorithm establishes the identification model of waste plastic;Using deformable convolution instead of ordinary convolution, using ResNeXt-101 as main network instead of ResNet-101 to improve feature extraction capability, establish final detection model;Based on the detection model trained, through camera shooting picture obtains final detection result.The application effectively solves the problem that waste plastic sorting background is complex and the problem that different plastic shape is greatly different, improves the identification precision of detecting multiple common waste plastic objects, and is helpful to practical production application.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of waste plastic detection and identification, and particularly relates to a waste plastic detection method based on an improved M2Det. BACKGROUND

[0002] Plastic products have been regarded as one of the "greatest" inventions of the 20th century. Today, with the rapid development of social economy, plastic products have become an indispensable part of people's lives. They have become a necessity in current society, whether in industry or agriculture. The domestic plastic product industry has therefore developed rapidly. Due to its excellent chemical stability, excellent wear resistance and formability, good insulation, low thermal conductivity, and low processing cost, plastic products have been widely used in production and manufacturing, transportation, national defense, medical treatment, and daily life. China is a major plastic producer. According to relevant industry reports, between 2011 and 2019, China's plastic product output showed an overall growth trend. In 2019, the output reached 8184.2 million tons, up 35.45% from 2018. China is also one of the major exporters of plastic products. In 2019, China exported 1424 million tons of plastic products, up 8.54% from 2018.

[0003] Plastics play an important role in various fields such as food packaging, construction industry, and industrial parts due to their low manufacturing cost, light weight, and wide range of applications. Currently, China's plastic production and sales rank first in the world. However, plastics also have the problems of wide distribution and difficult degradation, and every year there are inestimable amounts of plastic waste. Improper disposal of these plastic wastes can cause environmental pollution. China's traditional methods for disposing of waste plastic waste are mainly centralized landfill and incineration. These methods have some effect in the short term, but in the long term, they can cause serious pollution to the ecological environment. For example, it takes 400 years for a plastic bottle to decompose. These methods also do not meet the country's sustainable development strategy. Therefore, it is urgent to develop a healthy and sustainable waste plastic classification and recycling method, and classification of plastic waste is the key.

[0004] At present, the large-scale classification of waste plastics in China is still mainly manual, with poor working environment, high labor cost and low efficiency, which restricts the development of the industry. Traditional visual detection algorithms mostly require more manual pre-design, such as selecting appropriate sliding windows to slide on the image in order to extract the area where the object may exist, then using the pre-selected feature extraction operator such as HOG to extract features, and finally using shallow machine learning algorithms such as SVM for classification and regression prediction. Because the size of the target object is usually indefinite, traditional target detection algorithms need to design windows of various sizes, resulting in large amount of calculation and slow operation speed, and at the same time, the recognition accuracy is not high. In recent years, with the rapid improvement of the processing performance of CPU and DSP chips and the continuous exploration of computer vision related algorithms, computer vision technology has begun to be applied in industrial manufacturing and other fields. In the 21st century, deep learning technology has entered an explosive development stage. With the technical accumulation of deep learning in the field of vision, computer vision technology has achieved a great breakthrough and practical landing, and has achieved good results in many fields such as industry, agriculture, pharmaceuticals and services. The rapid rise of image processing, computer vision and artificial intelligence provides a new way for the sorting of waste plastics. With image processing, image feature extraction and target detection as technical support, the image features are obtained through the processing of waste plastic images, mainly the shape features of the images, and then the classification algorithm is designed according to the shape to realize the recognition of the images. However, there are still many problems in the visual sorting system at home and abroad at present, such as low sorting accuracy; for plastic species with large size span, it is difficult to identify, resulting in the need for secondary manual screening; objects inevitably block each other, leading to missed detection and other problems.

[0005] As mentioned in "Application of Decision Tree Algorithm in Plastic Classification", the method of laser irradiation can quickly and accurately identify the spectral lines of plastics. After obtaining the element composition and content of plastics by applying near-infrared spectroscopy technology, a plastic classification algorithm based on decision tree algorithm is proposed by using machine learning method with plastic composition as feature value, and finally the feature is selected by the principle of maximum entropy increase. Although the technology and method mentioned in this paper have very high detection accuracy, the premise is to accurately obtain the main components of plastics by laser irradiation, which requires the plastics to be pretreated before detection, including cleaning and other steps. In the actual application scenario, the environment of the recovered waste plastics is complex, with more surface sludge and various shapes, which undoubtedly has a great impact on laser irradiation, so the pretreatment of this method also increases the cost of waste plastic classification. SUMMARY

[0006] The existing waste plastic detection system is provided with an improved M2Det detection method, which is established on the basis of the basic M2Det method, and through the improvement of the M2Det target detection algorithm in multiple aspects, the improved algorithm can be quickly and efficiently applied to the waste plastic detection environment, and the algorithm is improved according to the actual scene of waste plastic, and then the detection precision and detection speed of waste plastic are improved. Moreover, the present application does not need high-cost pretreatment, only needs to scatter waste plastic, and is more widely applicable, and the present application can better identify the target category under the shielding condition.

[0007] The present application is realized by at least one of the following technical solutions.

[0008] An improved M2Det-based waste plastic detection method comprises the following steps:

[0009] An image data set is obtained, and data enhancement preprocessing is performed on the image data set;

[0010] An improved M2Det-based target detection model is constructed and trained;

[0011] The trained target detection model is used to detect the type of plastic samples.

[0012] Further, the improved M2Det-based target detection model uses deformable convolution instead of ordinary convolution to solve the problem of large target size difference, and uses ResNeXt-101 as the backbone network instead of ResNet-101 to improve the feature extraction capability.

[0013] Further, the backbone network uses VGG-16 and ResNeXt-101, and uses deformable convolution instead of ordinary convolution, and still uses the original MLFPN structure after the backbone network and introduces repulsive loss in the loss function.

[0014] Further, the improved M2Det-based target detection model comprises a backbone network and a multi-level feature pyramid network connected after the backbone network, and the multi-level feature pyramid network outputs detection in the manner of SSD.

[0015] Further, the backbone network comprises a first feature fusion module, a plurality of second feature fusion modules and a scale feature aggregation module.

[0016] The first feature fusion module fuses feature maps of different depths and sizes to obtain basic features of a uniform scale.

[0017] Stacking multiple refined U-shaped modules and second feature fusion modules, each refined U-shaped module generates multiple feature maps of different sizes, and each second feature fusion module fuses the basic features and the output of the last refined U-shaped module and gives it to the next refined U-shaped module as input;

[0018] The scale feature aggregation module splices the outputs of different refined U-shaped modules according to scales, and finally outputs a feature pyramid through an attention mechanism.

[0019] Further, the output of each level is as follows:

[0020]

[0021] wherein represents the i-th largest size feature map of the l-th refined U-shaped module, X base represents the basic feature map, T l represents the function of the l-th refined U-shaped module, and F represents the function of the feature fusion module FFM.

[0022] Further, the target detection model based on the improved M2Det is trained, including the following steps:

[0023] The preprocessed data set is labeled using Labelme software, and the training set and the test set are divided;

[0024] The training set is input into the target detection model based on the improved M2Det to start training, and the final training weight is obtained.

[0025] Further, the preprocessed image data set is divided into a training set, a validation set and a test set, the trained target detection model is used to detect the test set, and the mAP and the detection time are calculated as the final evaluation result.

[0026] Further, the preprocessing includes data enhancement and pixel transformation.

[0027] The data enhancement includes flipping, rotating, cropping, scaling, shifting, and jittering.

[0028] The pixel transformation includes adding salt and pepper noise, Gaussian noise, Gaussian blur, adjusting HSV contrast, adjusting brightness, saturation, histogram equalization, and adjusting white balance.

[0029] Further, the preprocessing includes Mosaic data enhancement.

[0030] Compared with the existing technology, the beneficial effects of the present application are:

[0031] The accuracy and speed of target detection and identification are improved to a certain extent, and the effective receptive field is improved on the basis of the original through deformable convolution. Finally, the repulsive loss is introduced in the loss function to effectively reduce the missed detection rate of the occluded target, which solves the problem of waste plastic detection under serious occlusion to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 A waste plastic detection method based on an improved M2Det is provided for the embodiment of the application, and a flowchart of the method is shown in the figure.

[0033] Figure 2 A target detection model structure diagram based on the improved M2Det is provided for the embodiment of the application.

[0034] Figure 3 A ResNeXt-101 block structure diagram is provided for the embodiment of the application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely in combination with the drawings of the present application. The described embodiments are only some of the embodiments of the present application, not all examples. Based on the examples in the present application, other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0036] As shown in the figure, a waste plastic detection method based on an improved M2Det includes the following steps: Figure 1

[0037] Step S1, obtaining a data set:

[0038] The picture sample collection of the embodiment mainly has two sources, the first part is the pictures taken offline in daily life, and the second part is collected by network crawler software on App such as Taobao, Jingdong, Baidu and 360 browsers, and the pictures obtained by the crawler are manually cleaned, and after screening out the pictures that do not meet the standard, a total of 6230 pictures are collected, and the number of various pictures is respectively 1704 beverage bottles, 1993 toiletry bottles, 1226 Tetra Pak boxes, 696 express packaging bags and 611 disposable tableware boxes. After obtaining the data set, all the data sets are labeled using the labelme software.

[0039] ​Step S2, in order to avoid overfitting, improve the robustness and generalization ability of the model. The data set needs to be preprocessed by data augmentation. The commonly used traditional data augmentation methods mainly include: flipping, rotating, cropping, scaling, shifting, pixel transformation methods: adding pepper and salt noise, Gaussian noise, Gaussian blur, adjusting HSV contrast, adjusting brightness, saturation, histogram equalization, adjusting white balance, etc.

[0040] As a preferred embodiment, in addition to the traditional data augmentation, a data augmentation method Mosaic data augmentation improved from CutMix is also used. The CutMix data augmentation method in the field of object detection is to randomly paste a training image on another image after random cutting and scaling, and handle the label boundary. First, get the first image according to the index, then randomly select another three images for splicing, which enriches the training background. Due to the limitation of model size and memory, the model can only take a smaller batch of training images during training. Mosaic data augmentation is equivalent to training four images at a time, and the batch number is increased by three times. It not only enriches the data samples but also increases the training batch number. Using Mosaic data augmentation can often achieve stable growth points. Compared with traditional data augmentation, it is more powerful and effective.

[0041] As a preferred embodiment, the processed images are uniformly sized to 320x320.

[0042] Step S3, the preprocessed and converted waste plastic image data set is randomly divided into a training set, a validation set and a test set according to a ratio of 7:1.5:1.5, wherein the training set is used to find the optimal hyperparameters, the trained object detection model is used to detect the test set, and the mAP and detection time are calculated as the final evaluation results. The mAP is an average precision mean, which considers the classification and positioning accuracy of the model, and is also the most widely used model detection precision evaluation index. The detection speed is detected by measuring the model inference speed index FPS (Frame Per Second, frame rate per second). Generally, it is converted by measuring the time required for the model to process the same image multiple times.

[0043] Step S4, the improved M2Det model is obtained based on the improved M2Det target detection model: a deeper and wider variant ResNeXt-101 is used as the backbone network instead of ResNet-101, which can improve the feature extraction ability and accuracy of the model, ResNeXt-101 is an expansion of ResNet-101 in width, ResNeXt has lower error rate than ResNet during training under the same number of parameters, but the decline speed is almost the same, and the 101-layer ResNeXt has better effect than the 200-layer ResNet, the performance of several sota models, the accuracy and performance of ResNeXt are obviously better than those of ResNet; the deformable convolution with scaling factor is used instead of the ordinary convolution, the offset of each sampling point is added, and then the offset is iteratively optimized and learned by using the back propagation algorithm, so as to improve the effective receptive field of the convolution process; the repulsion loss is introduced based on the original loss function, which can reduce the loss between the predicted box and the real box, and increase the loss between the predicted box and the real box around the target, reduce the missed detection rate of the occluded target, and make the target detector have higher detection accuracy. The improved M2Det composition structure is as follows: the VGG-16 and ResNeXt-101 are used as the backbone network, the original MLFPN structure is still used after the backbone network, the deformable convolution is used instead of the ordinary convolution, and the repulsion loss is introduced in the loss function.

[0044] Step S5, the improved M2Det target detection model is iteratively trained, first, the training set is input into the network model to start training, the training device used is an eighteen-core CPU with i9-10980XE, 48GB of video memory and two Nvidia graphics cards of RTX3090 type, the operating system is an Ubuntu server, the network is set to train a total of 100 rounds, and the final training weight is obtained after the structure training of the network model.

[0045] The M2Det is used as the basic target detection algorithm in the application, and the M2Det algorithm is briefly described below.

[0046] M2Det is an article published by Peking University in AAAI in 2019, which belongs to one-stage network, in order to more effectively solve the problem of scale change in target detection, it creatively puts forward a more effective feature pyramid structure-MLFPN structure. MLFPN is established on the basis of FPN, and a small FPN module is used to replace the layers of different depths in the original FPN framework. At the same time, the SE block is introduced, and different depth features are given weights.

[0047] The general process is shown in the figure: first, the features extracted from the backbone network are fused; then more representative multi-level and multi-scale features are extracted through the detailed U-shaped module (TUM) and the feature fusion module (FFM); finally, the multi-level features are fused through the scaling feature aggregation module (SFAM) to obtain a multi-level feature pyramid for the final prediction.

[0048] As shown in Figure 2 , the target detection model improved from M2Det includes:

[0049] The backbone directly uses VGG-16 (Visual Geometry Group Network) and Resnet-101 (Deep Residual Neural Network). The MLFPN (Multi-Level Feature Pyramid Network) module is connected after the backbone, and the detection out uses the SSD (Single Shot MultiBox Detector) method.

[0050] First, the FFMv1 (first feature fusion module) submodule is responsible for fusing feature maps of different depths and sizes in the backbone to obtain a base feature of a unified scale.

[0051] Second, multiple TUMs (detailed U-shaped modules) and FFMv2s (second feature fusion modules) are stacked, each TUM can produce multiple feature maps of different scales, and each FFMv2 fuses the base feature and the output of the previous TUM and gives it to the next TUM as input (higher level). The output of each level is described in the following formula;

[0052]

[0053] Where represents the feature map of the ith largest size of the lth layer TUM (in ascending order of size), X base represents the base feature map, T l represents the processing process of the lth TUM, and F represents the processing process of the FFM.

[0054] Finally, the SFAM (Scale Feature Aggregation Module) module is responsible for splicing the outputs of different TUM modules by scale, and passing through a SE block (attention mechanism, Squeeze-and-Excitation), and finally outputting a feature pyramid. The reason why M2Det can exceed the SOTA (the most advanced network in the industry) is the MLFPN. On the one hand, the multi-layer feature maps in the backbone are fused as basic features, and then multi-layer multi-level feature maps are extracted from the basic features for target detection, which is more representative than FPN which only uses single-layer features for target detection. Both the receptive field of the feature map and the semantic information of different layers are considered. Compared with the image pyramid which only considers the scale level, and compared with the feature hierarchy which only considers the level, FPN and MLFPN are more reasonable, but FPN mainly considers the scale level, and MLFPN considers both.

[0055] However, the existing M2Det model cannot be completely applied to the waste plastic detection scene, the first reason is that the waste plastic detection background is relatively complex, in the actual application scene, even after preliminary screening, other various garbage debris will be mixed in, at the same time, the waste plastic will also exist damage and deformation, etc., so that the appearance sometimes will change greatly, it is difficult to detect, therefore, the model needs excellent feature extraction capability; the second reason is that the shapes of some categories of waste plastics are quite different, the multi-scale problem leads to insufficient generalization ability of the model, so that the detection effect is poor; the third reason is that the waste plastics will inevitably appear mutual occlusion problem, which leads to error in model recognition.

[0056] Step S6, test the model, use the trained model to predict the category of samples in the test set, and calculate mAp and FPS as the final evaluation result.

[0057] Step S7, identify the type of waste plastic through the trained target detection model. The weight file after training is imported into the prediction code, and the prediction code performs real-time detection on the waste plastic.

[0058] In order to improve the feature extraction capability of the model in the complex sorting background, a deeper and wider variant ResNeXt-101 is used as the backbone network instead of ResNet-101. ResNeXt-101 introduces the multi-branch idea of Inception, and changes the single convolution of ResNet-101 into multi-branch convolution, such as Figure 3The block structure diagram of the shown ResNeXt: the input channel number is reduced to 4 channels, then a 3x3 convolution is connected without changing the channel number, and finally the channel number is restored to 256. Then the output tensors of 32 branches are summed, and finally the input is added to the result by using the cross-layer connection. ResNeXt-101 continues to improve the accuracy while maintaining high portability. The MLFPN of M2Det itself can effectively solve the multi-scale problem, in order to be more accurate, the application uses deformable convolution with scaling factor instead of ordinary convolution, adds the offset of each sampling point, and then uses the back propagation algorithm to iteratively optimize the learning offset, so as to improve the effective receptive field of the convolution process. The repulsion loss is used in the original loss function to reduce the missed detection rate of the occluded target.

[0059] The preferred embodiments of the application disclosed above are only used to help explain the application. The preferred embodiments do not describe all the details and limit the application to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the specification. The specification selects and describes these embodiments in order to better explain the principles and practical applications of the application, so that those skilled in the art can well understand and use the application. The application is limited by the claims and their full scope and equivalents.

Claims

1. A waste and miscellaneous plastic detection method based on improved M2Det, characterized by, The method comprises the following steps: An image data set is acquired, and data augmentation preprocessing is performed on the image data set; An improved M2Det-based target detection model is constructed and trained; The improved M2Det-based target detection model is used to detect the type of a plastic sample; The improved M2Det-based target detection model uses deformable convolution instead of ordinary convolution to solve the problem of large differences in target size, and uses ResNeXt-101 as a backbone network instead of ResNet-101 to improve feature extraction capability; The backbone network uses VGG-16 and ResNeXt-101, and uses deformable convolution instead of ordinary convolution; the original MLFPN structure is still used after the backbone network, and repulsion loss is introduced in the loss function; The improved M2Det-based target detection model comprises a backbone network and a multi-level feature pyramid network connected after the backbone network, and the multi-level feature pyramid network outputs detection in the manner of SSD; The backbone network comprises a first feature fusion module, a plurality of second feature fusion modules, and a scale feature aggregation module; The first feature fusion module fuses feature maps of different depths and sizes to obtain basic features of a uniform scale; A plurality of refining U-shaped modules and second feature fusion modules are stacked, each refining U-shaped module generates a plurality of feature maps of different sizes, and each second feature fusion module fuses the basic features and the output of the previous refining U-shaped module and provides the output to the next refining U-shaped module as input; The scale feature aggregation module splices the outputs of different refining U-shaped modules according to scales, and finally outputs a feature pyramid through an attention mechanism.

2. The method for detecting waste and miscellaneous plastics based on improved M2Det according to claim 1, characterized in that, The output of each level is described by the following formula; wherein denotes the layer of the feature maps of large size, denotes the base feature maps, denotes the function of the function of the feature fusion module FFM.

3. The method for detecting waste and miscellaneous plastics based on improved M2Det according to claim 1, characterized in that, Training the improved M2Det-based target detection model comprises the following steps: Labeling is performed on the preprocessed data set using Labelme software, and a training set and a test set are divided; The training set is input into the improved M2Det-based target detection model to start training, and final training weights are obtained.

4. The method for detecting waste and miscellaneous plastics based on improved M2Det according to claim 1, characterized in that, The preprocessed image data set is divided into a training set, a validation set, and a test set; the trained target detection model is used to detect the test set, and mAP and detection time are calculated as the final evaluation result.

5. The improved M2Det-based waste and miscellaneous plastic detection method according to any one of claims 1-4, characterized in that, The preprocessing comprises data augmentation and pixel transformation; The data augmentation comprises flipping, rotating, cropping, scaling, panning, and jittering; The pixel transformation comprises adding salt and pepper noise, adding Gaussian noise, performing Gaussian blur, adjusting HSV contrast, adjusting brightness, saturation, histogram equalization, and adjusting white balance.

6. The method for detecting waste and miscellaneous plastics based on improved M2Det according to claim 5, characterized in that, The preprocessing comprises Mosaic data augmentation.

Images