A garbage classification method integrating SVM and target detection algorithm
Through the integration of SVM and YOLOv4 object detection algorithm, combined with hyperspectral camera and PLS dimensionality reduction model, the problem of low recognition accuracy of garbage sorting robots is solved, and more efficient garbage classification and recognition is achieved, reducing labeling complexity and data redundancy. Therefore, the identification accuracy has been improved.
Patent Information
- Application Number
- CN202111323651.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-10
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-11-10
AI Technical Summary
The existing garbage sorting robot image recognition technology has low recognition accuracy in garbage classification. Model establishment requires a large number of image markings, and due to uncertainty in production lines, the phenomenon of empty grabbing and leakage grabbing is frequent.
Using the method of integrating SVM and YOLOv4 object detection algorithm, spectral data are collected through hyperspectral cameras, features are extracted using PLS dimensionality reduction model, RGB images are generated, and SVM classification model and YOLOv4 object detection model are trained and detected to reduce labeling complexity and data redundancy to improve identification accuracy.
It improves the accuracy of garbage identification, reduces the incidence of empty grab and miss grab, simplifies the labeling process, and improves the computing speed and accuracy of the model.
Smart Images

Figure CN114049556B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a garbage classification method, in particular to a garbage classification method integrating SVM with a target detection algorithm, and belongs to the field of garbage classification. Background Art
[0002] Garbage classification refers to the storage, placement and transportation of garbage according to certain regulations or standards, thus transforming it into a public resource. The traditional method of garbage classification mainly uses manual sorting, but the sorting efficiency is low. Therefore, many sorting centers now use sorting robots for sorting. Sorting robots not only have high disposal efficiency, but also use unmanned operation, which greatly improves the health and safety of sorting workers.
[0003] Image recognition, or object detection, is an important part of the garbage sorting robot. As the "eyes" of the sorting robot, its recognition accuracy determines the sorting efficiency of the sorting robot. At present, the image recognition (object detection) of sorting robots mostly adopts deep learning methods. As a new field of machine learning, deep learning has achieved success in various fields. However, in the field of garbage sorting, image recognition (object detection) is almost always developed based on frameworks such as YOLO, FasterRCNN, MaskRCNN, and YOLACT. Its disadvantage is that in the early stage of model establishment, a large number of picture annotations are required to improve the accuracy of the model. In addition, due to the uncertainty of incoming materials on the production line, if the development is based solely on the above framework, the probability of empty grasping and missed grasping by the sorting robot will be greatly increased. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a garbage classification method integrating SVM with target detection algorithm, so as to improve the accuracy of identifying garbage to be classified.
[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0006] A garbage classification method integrating SVM and target detection algorithm, characterized by comprising the following steps:
[0007] Step 1: Collect spectral data of classified garbage through a hyperspectral camera;
[0008] Step 2: Reduce the dimension of the collected spectral data through the PLS dimension reduction model;
[0009] Step 3: Use the SVM classification model to classify the spectral data, and color the classified spectral data with different colors according to different materials to generate an RGB image;
[0010] Step 4: Use the YOLOv4 target detection model to train and detect the generated RGB images.
[0011] Furthermore, the step one is specifically that the hyperspectral camera scans the material spectral information of the garbage to be classified by line scanning, and the spectral band range of the scan is 400-1000nm or 900-1700nm.
[0012] Furthermore, the step 2 is specifically as follows:
[0013] 2.1 The spectral data of the garbage to be classified collected by the hyperspectral camera is a two-dimensional information matrix of the garbage to be classified;
[0014] 2.2 The partial least squares regression method is used to reduce the dimension of the spectral data of the classified garbage using the PLS dimension reduction model. The formula of the PLS dimension reduction model is:
[0015] A=TP T +E (1)
[0016] B=UQ T +F (2)
[0018] Where A represents the n*m prediction matrix; B represents the n*p response matrix; T and U represent n*l matrices, which are projections of A and B respectively; P and Q represent m*l and p*l orthogonal loading matrices; matrices E and F are error terms; formulas (1) and (2) are operated together to complete the dimensionality reduction of the PLS model;
[0019] 2.3 After the spectral data of the garbage to be classified is reduced in dimension by the PLS dimensionality reduction model, feature extraction is performed from more than 200 mixed spectral pixel information, and the bands or pixel information containing large amounts of information are selected to reduce the data redundancy. Two characteristic spectral values are extracted and recorded as the values of coordinates X and Y respectively. After all spectral data are reduced in dimension by the PLS dimensionality reduction model, a scatter plot of the garbage to be classified is obtained.
[0020] Furthermore, the SVM classification model is used to classify the spectral data in step 3 as follows:
[0021] 3.1 After obtaining the scatter plot, use the multi-classifier in the support vector machine SVM model to separate different pixels, that is, assume that one is a positive sample and the rest are negative samples. The point-to-plane distance of any sample must be ≥ 1. The decision boundary and point-to-plane distance formula are as follows:
[0022] Decision Boundary: ω T X+b=0
[0023] Distance from point to plane: y i (ω T Xi +b)≥1
[0024] Among them, the parameters w and b are the normal vector and intercept of the hyperplane respectively; X and y are the sample and label class respectively;
[0025] 3.2 Use the standardized training set samples to train the SVM multi-classifier, use the cross-validation idea to find the optimized parameters C and γ, and optimize the classifier; where the parameter γ is the γ parameter of the polynomial, Gaussian radial basis, and two-layer perceptron kernel function, and the parameter C is the SVM penalty parameter, which is a non-negative number;
[0026] 3.3 By repeating steps 3.1-3.2, all kinds of samples are separated and a classification model is obtained.
[0027] Furthermore, if the coordinate point information of the garbage to be classified is in the middle area of the three lines, classification is achieved through the weight during classification, and the classification model is now completed.
[0028] Furthermore, when analyzing a new spectrum, the spectral data is passed through the PLS dimension reduction model to obtain X and Y coordinate values. The coordinate values can classify the spectrum through the PLS dimension reduction model, thereby classifying each collected pixel point.
[0029] Furthermore, in step 3, the classified spectral data is colored with different colors according to different materials to generate an RGB image, specifically:
[0030] 3.4 For each pixel after classification, segment it according to the different materials corresponding to the pixel value of the pixel, and then assign a specified type of color to each pixel according to the different materials corresponding to the segmentation information;
[0031] 3.5 Splice the pixel color information of 416 lines of each line scan into a 416*416 RGB pseudo-color image.
[0032] Furthermore, the step 4 is specifically as follows:
[0033] 4.1 Label the obtained RGB pseudo-color images of the garbage to be classified, and establish a sample data set with the labeled RGB pseudo-color images;
[0034] 4.2 Put the sample data set into the YOLOv4 target detection model according to 416*416 for training;
[0035] 4.3 Input the pseudo-color image into the YOLOv4 target detection model with an input size of 416*416 to obtain a list of all the bounding boxes of the garbage to be classified in the image, and filter them through the non-maximum suppression NMS algorithm to obtain the coordinate position information of the target garbage points that need to be retained; the non-maximum suppression NMS algorithm is as follows:
[0036]
[0037] Among them, Si represents the score of each border, M represents the box with the highest current score, bi represents a box among the remaining boxes, Nt is the set NMS threshold, and iou is the ratio of the overlapping areas of the two recognition boxes;
[0038] 4.4 Send the point information and type predicted by the YOLOv4 target detection model to the robot for grasping.
[0039] Furthermore, the above 4.2 is specifically as follows:
[0040] CIOU is used to calculate the loss during the training of the YOLOv4 target detection model. The bbox regression in CIOU can solve the three problems of coverage area, center point distance and aspect ratio in the identification process of the garbage to be classified. The calculation formula of bbox regression is as follows:
[0041] LOSS_CIOU=1-IOU+(ρ 2 *(b,b gt ) / c 2 )+αυ
[0042] Among them, (ρ 2 *(b,b gt ) is the Euclidean distance between the center points of the predicted box and the true box; c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box; the expansion formula of α and v are as follows:
[0043] α=υ / (1-IOU+υ)
[0044] υ=(4 / π 2 )*(arctan(w gt / h gt )-arctan(w / h)) 2 .
[0045] Compared with the prior art, the present invention has the following advantages and effects:
[0046] 1. The present invention uses a hyperspectral camera to obtain spectral data of the garbage to be classified. Compared with the prior art that uses an ordinary industrial 2D camera to collect the three RGB information values of each pixel of the image, the spectral data of the present invention contains 200 information values for each pixel, and the obtained pixel material information is more accurate. The spectrum covers the main band range of conventional items, and can still accurately classify uncertain incoming materials, thereby improving the accuracy of identifying the garbage to be classified and solving the problems of empty grasping and missed grasping of the sorting robot in the prior art.
[0047] 2. The present invention uses a hyperspectral camera to collect spectral data and generate RGB pseudo-color images. Compared with the existing detection algorithm that requires each target to be labeled, the present invention only needs to label the material represented by the spectrum, which is less complex and reduces the difficulty of labeling;
[0048] 3. The present invention reduces the dimension of spectral data through the PLS dimension reduction model, extracts feature data, selects bands or features containing large amounts of information to reduce data redundancy, improves the speed of model operation, and increases the accuracy of the model;
[0049] 4. The present invention classifies different spectra through the SVM model, which is simple, accurate and more robust. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 It is a flow chart of a garbage classification method integrating SVM and target detection algorithm of the present invention.
[0051] Figure 2 It is a training flow chart of the YOLOv4 target detection model of the present invention.
[0052] Figure 3 It is a detection flow chart of the YOLOv4 target detection model of the present invention.
[0053] Figure 4 It is a spectrum information diagram of three categories of materials obtained by scanning with a hyperspectral camera in an embodiment of the present invention.
[0054] Figure 5 It is a scatter plot of materials of three categories in an embodiment of the present invention.
[0055] Figure 6 It is a pseudo color map of three categories of materials in an embodiment of the present invention.
[0056] Figure 7 It is a block diagram of a material target identified by a YOLOv4 target detection model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0057] In order to elaborate on the technical scheme adopted by the present invention to achieve the predetermined technical purpose, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only partial embodiments of the present invention, rather than all embodiments, and the technical means or technical features in the embodiments of the present invention can be replaced without paying creative work. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.
[0058] like Figure 1As shown, a garbage classification method integrating SVM and target detection algorithm of the present invention comprises the following steps:
[0059] Step 1: Collect spectral data of classified garbage through a hyperspectral camera; the hyperspectral camera scans the material spectral information of the garbage to be classified by line scanning, and the spectral band range of the scan is 400-1000nm or 900-1700nm. The spectral band range of conventional items is mainly 400-1000nm or 900-1700nm, which is widely used in industrial sorting. It can perform non-destructive detection of defective products with the same composition or shape, surface or internal deterioration, and various types of adulteration in large quantities of mixed materials, and the recognition speed is fast, the fastest can reach 670Hz.
[0060] Step 2: Reduce the dimension of the collected spectral data through the PLS dimension reduction model; extract the feature data and select the bands or features containing large amounts of information to reduce the redundancy of the data, thereby improving the model operation speed and increasing the model accuracy.
[0061] 2.1 The spectral data of the garbage to be classified collected by the hyperspectral camera is a two-dimensional information matrix of the garbage to be classified;
[0062] 2.2 The partial least squares regression method is used to reduce the dimension of the spectral data of the classified garbage using the PLS dimension reduction model. The formula of the PLS dimension reduction model is:
[0063] A=TP T +E (1)
[0064] B=UQ T +F (2)
[0065] Where A represents the n*m prediction matrix; B represents the n*p response matrix; T and U represent n*l matrices, which are projections of A and B respectively; P and Q represent m*l and p*l orthogonal loading matrices; matrices E and F are error terms; formulas (1) and (2) are operated together to complete the dimensionality reduction of the PLS model;
[0066] 2.3 After the spectral data of the garbage to be classified is reduced in dimension by the PLS dimensionality reduction model, feature extraction is performed from more than 200 mixed spectral pixel information, and the bands or pixel information containing large amounts of information are selected to reduce the data redundancy. Two characteristic spectral values are extracted and recorded as the values of coordinates X and Y respectively. After all spectral data are reduced in dimension by the PLS dimensionality reduction model, a scatter plot of the garbage to be classified is obtained.
[0067] Step 3: Use the SVM classification model to classify the spectral data, and color the classified spectral data with different colors according to different materials to generate an RGB image; the colored color is a self-named pseudo color, which is mainly used to distinguish the spectral pixel information of different materials.
[0068] 3.1 After obtaining the scatter plot, use the multi-classifier in the support vector machine SVM model to separate different pixels, that is, assume that one is a positive sample and the rest are negative samples. The point-to-plane distance of any sample must be ≥ 1. The decision boundary and point-to-plane distance formula are as follows:
[0069] Decision Boundary: ω T X+b=0
[0070] Distance from point to plane: y i (ω T X i +b)≥1
[0071] Among them, the parameters w and b are the normal vector and intercept of the hyperplane respectively; X and y are the sample and label class respectively;
[0072] 3.2 Use the standardized training set samples to train the SVM multi-classifier, use the cross-validation idea to find the optimized parameters C and γ, and optimize the classifier; where the parameter γ is the γ parameter of the polynomial, Gaussian radial basis, and two-layer perceptron kernel function, and the parameter C is the SVM penalty parameter, which is a non-negative number;
[0073] 3.3 By repeating steps 3.1-3.2, all kinds of samples are separated and a classification model is obtained.
[0074] If the coordinate point information of the garbage to be classified is in the middle area of the three lines, the classification is achieved through the weight during classification, and the classification model is now completed.
[0075] When analyzing a new spectrum, the spectral data is passed through the PLS dimension reduction model to obtain X and Y coordinate values. The coordinate values can be used to classify the spectrum through the PLS dimension reduction model, thereby classifying each collected pixel point.
[0076] 3.4 For each pixel after classification, segment it according to the different materials corresponding to the pixel value of the pixel, and then assign a specified type of color to each pixel according to the different materials corresponding to the segmentation information;
[0077] 3.5 As Figure 2 As shown, the pixel color information of 416 rows in each line scan is spliced into an RGB pseudo-color image of size 416*416.
[0078] Step 4: Use the YOLOv4 target detection model to train and detect the generated RGB images.
[0079] 4.1 As Figure 3 As shown, the obtained RGB pseudo-color images of the garbage to be classified are annotated, and a sample data set is established with the annotated RGB pseudo-color images;
[0080] 4.2 Put the sample data set into the YOLOv4 target detection model according to 416*416 for training;
[0081] CIOU is used to calculate the loss during the training of the YOLOv4 target detection model. The bbox regression in CIOU can solve the three problems of coverage area, center point distance and aspect ratio in the identification process of the garbage to be classified. The calculation formula of bbox regression is as follows:
[0082] LOSS_CIOU=1-IOU+(ρ 2 *(b,b gt ) / c 2 )+αυ
[0083] Among them, (ρ 2 *(b,b gt ) is the Euclidean distance between the center points of the predicted box and the true box; c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box; the expansion formula of α and v are as follows:
[0084] α=υ / (1-IOU+υ)
[0085] υ=(4 / π 2 )*(arctan(w gt / h gt )-arctan(w / h)) 2 .
[0086] After training in the YOLOv4 target detection model, the model recognition accuracy can be effectively improved and the accuracy of the carbon value estimation of the target material can be increased.
[0087] 4.3 Input the pseudo-color image into the YOLOv4 target detection model with an input size of 416*416 to obtain a list of all the bounding boxes of the garbage to be classified in the image, and filter them through the non-maximum suppression NMS algorithm to obtain the coordinate position information of the target garbage points that need to be retained; the non-maximum suppression NMS algorithm is as follows:
[0088]
[0089] Among them, Si represents the score of each border, M represents the box with the highest current score, bi represents a box among the remaining boxes, Nt is the set NMS threshold, and iou is the ratio of the overlapping areas of the two recognition boxes;
[0090] 4.4 Send the point information and type predicted by the YOLOv4 target detection model to the robot for grasping.
[0091] The present application is further described below in conjunction with specific embodiments.
[0092] 1) Use a hyperspectral line scan camera to scan three different types of plastic bottles (PET, PP, HDPE) Figure 4 As shown, the blue wavy line represents HDPE, the green wavy line represents PP, and the red wavy line represents PET;
[0093] 2) If Figure 5 As shown in the figure, the PLS model is used for classification to obtain scatter plots of three different types of plastic bottles. It can be found that after the dimensionality reduction by the PLS model, the spectral feature band information is extracted, and a scatter plot with more distinct features is obtained, and the classification effect is good;
[0094] 3) If Figure 6 As shown in the figure, the pixel scatter diagrams of three different colors are spliced into a pseudo color map of 416*416 size, and the plastic bottles of three different materials, red, green and blue, can be clearly seen in the figure;
[0095] 4) If Figure 7 As shown in the figure, the YOLOv4 target detection model is used to obtain the target object box list, and the coordinate information of the target material is obtained after filtering by the NMS algorithm. This method has better capture effect and higher accuracy.
[0096] It should be noted that the present invention is attached Figure 4-7 The red, green, and blue text labels are only for the convenience of color distinction in black and white images, and are not necessary information in the image processing process.
[0097] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.
Claims
1. A garbage classification method integrating SVM and target detection algorithm, characterized in that The following steps are involved: Step 1: Collect spectral data of classified garbage through a hyperspectral camera; Step 2: Reduce the dimension of the collected spectral data through the PLS dimension reduction model; The step 2 is specifically as follows: 2.1 The spectral data of the garbage to be classified collected by the hyperspectral camera is a two-dimensional information matrix of the garbage to be classified; 2.2 The partial least squares regression method is used to reduce the dimension of the spectral data of the classified garbage using the PLS dimension reduction model. The formula of the PLS dimension reduction model is: A=TP T +E (1) B=UQ T +F (2) Where A represents the n*m prediction matrix; B represents the n*p response matrix; T and U represent n*l matrices, which are projections of A and B respectively; P and Q represent m*l and p*l orthogonal loading matrices; matrices E and F are error terms; formulas (1) and (2) are operated together to complete the dimensionality reduction of the PLS model; 2.3 After the spectral data of the garbage to be classified is reduced in dimension by the PLS dimensionality reduction model, feature extraction is performed from more than 200 mixed spectral pixel information, and the bands or pixel information containing large amounts of information are selected to reduce the data redundancy, and two characteristic spectral values are extracted, which are recorded as the values of coordinates X and Y respectively. After all spectral data are reduced in dimension by the PLS dimensionality reduction model, a scatter plot of the garbage to be classified is obtained; Step 3: Use the SVM classification model to classify the spectral data, and color the classified spectral data with different colors according to different materials to generate an RGB image; In step 3, the classified spectral data is colored with different colors according to different materials to generate an RGB image, specifically: 3.4 For each pixel after classification, segment it according to the different materials corresponding to the pixel value of the pixel, and then assign a specified type of color to each pixel according to the different materials corresponding to the segmentation information; 3.5 Splice the pixel color information of 416 lines of each line scan into a 416*416 RGB pseudo-color image; Step 4: Use the YOLOv4 target detection model to train and detect the generated RGB images.
2. The garbage classification method integrating SVM and target detection algorithm according to claim 1 is characterized in that: The step one is specifically that the hyperspectral camera scans the material spectral information of the garbage to be classified by line scanning, and the spectral band range of the scan is 400-1000nm or 900-1700nm.
3. The garbage classification method integrating SVM and target detection algorithm according to claim 1 is characterized in that: In step 3, the SVM classification model is used to classify the spectral data as follows: 3.1 After obtaining the scatter plot, use the multi-classifier in the support vector machine SVM model to separate different pixels, that is, assume that one is a positive sample and the rest are negative samples. The point-to-plane distance of any sample must be ≥ 1. The decision boundary and point-to-plane distance formula are as follows: Decision Boundary: ω T X i +b=0 Distance from point to plane: y i (ω T X i +b)≥1 Among them, the parameters ω and b are the normal vector and intercept of the hyperplane respectively; X i ,y i They are samples and label classes respectively; 3.2 Use the standardized training set samples to train the SVM multi-classifier, use the cross-validation idea to find the optimized parameters C and γ, and optimize the classifier; where the parameter γ is the γ parameter of the polynomial, Gaussian radial basis, and two-layer perceptron kernel function, and the parameter C is the SVM penalty parameter, which is a non-negative number; 3.3 By repeating steps 3.1-3.2, all kinds of samples are separated and a classification model is obtained.
4. The garbage classification method integrating SVM and target detection algorithm according to claim 1 is characterized in that: If the coordinate point information of the garbage to be classified is in the middle area of the three lines, the classification is achieved through the weight during classification, and the classification model is now completed.
5. The garbage classification method integrating SVM and target detection algorithm according to claim 1 is characterized in that: When analyzing a new spectrum, the spectral data is passed through the PLS dimension reduction model to obtain X and Y coordinate values. The coordinate values can be used to classify the spectrum through the PLS dimension reduction model, thereby classifying each collected pixel point.
6. The garbage classification method integrating SVM and target detection algorithm according to claim 1 is characterized in that: The step 4 is specifically as follows: 4.1 Label the obtained RGB pseudo-color images of the garbage to be classified, and establish a sample data set with the labeled RGB pseudo-color images; 4.2 Put the sample data set into the YOLOv4 target detection model according to 416*416 for training; 4.3 Input the pseudo-color image into the YOLOv4 target detection model with an input size of 416*416 to obtain a list of all the bounding boxes of the garbage to be classified in the image, and filter them through the non-maximum suppression NMS algorithm to obtain the coordinate position information of the target garbage points that need to be retained; the non-maximum suppression NMS algorithm is as follows: Among them, S i Represents the score of each box, M represents the box with the highest current score, and b i Represents a box among the remaining boxes, N t is the set NMS threshold, iou() is the overlap area ratio of the two recognition boxes; 4.4 Send the point information and type predicted by the YOLOv4 target detection model to the robot for grasping.
7. The garbage classification method integrating SVM and target detection algorithm according to claim 6 is characterized in that: The 4.2 is specifically: CIOU is used to calculate the loss during the training of the YOLOv4 target detection model. The bbox regression in CIOU can solve the three problems of coverage area, center point distance and aspect ratio in the identification process of the garbage to be classified. The calculation formula of bbox regression is as follows: LOSS_CIOU=1-IOU+(ρ 2 *(b,b gt ) / c 2 )+αv Among them, (ρ 2 *(b,b gt ) is the Euclidean distance between the center points of the predicted box and the true box; c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box; the expansion formula of α and v are as follows: α=v / (1-IOU+v) v=(4 / π 2 )*(arctan(ω gt / h gt )-arctan(ω / h)) 2 。
