A method for estimating garbage carbon value based on deep learning and hyperspectral recognition technology

Abstract

The present invention discloses a garbage carbon value estimation method based on deep learning and hyperspectral recognition technology. A 2D industrial camera takes RGB images of garbage, and a hyperspectral line scan camera synchronously collects hyperspectral image information of garbage; the RGB image is input into the YOLOv4 target detection model to obtain the image and filter it through the NMS algorithm to obtain the coordinate position information of the target garbage point; the hyperspectral image information is reduced in dimension by the PLS model and classified; the YOLOv4 target detection model is traversed to predict the target frame confidence, and it is judged whether each target frame confidence is greater than 0.8, and the final carbon value of the target object is output. The present invention provides a garbage carbon value estimation method based on the combination of deep learning and hyperspectral recognition technology, which can perform carbon value analysis and calculation on incoming garbage, improve garbage disposal efficiency, and enhance the accuracy of recognition calculation.

Description

Technical Field

[0001] The present invention relates to a carbon value estimation method, in particular to a garbage carbon value estimation method based on deep learning and hyperspectral recognition technology, belonging to the technical field of intelligent garbage processing. Background Art

[0002] At present, the final treatment of domestic waste in my country is mainly landfill and incineration. Among them, incineration is a common way to treat domestic waste. In order to promote the combustion effect, some fossil fuels such as coal fossils and fuel oil are usually added to the incineration of domestic waste. These fuels will produce a certain amount of carbon dioxide gas as the waste is incinerated, and carbon emissions will increase accordingly. Although domestic waste landfill can bury the waste underground, a certain amount of carbon-containing gas will also be produced in the process. The leachate used in the landfill will also release nitric oxide and other substances containing carbon elements in the leachate regulating tank. Therefore, the current methods of terminal disposal of domestic waste will inevitably produce carbon dioxide during the disposal process.

[0003] In order to promote the reduction of carbon dioxide emissions, low-carbon disposal of garbage is an effective way to reduce carbon dioxide emissions. Estimating the carbon value of garbage can promote low-carbon disposal of garbage to a certain extent, providing a new idea for reducing carbon dioxide emissions. However, the traditional garbage carbon value estimation method only uses manual methods to judge the material of the material, which has large uncertainty factors, and many materials on the market are often mixed with other materials and have low purity, which greatly reduces the accuracy of garbage carbon value estimation. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology, which can automatically estimate the carbon value of incoming garbage with high estimation accuracy.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology, characterized by comprising the following steps:

[0007] Step 1: The 2D industrial camera takes RGB images of the garbage, and the hyperspectral line scan camera simultaneously collects the hyperspectral image information of the garbage;

[0008] Step 2: Input the RGB image into the YOLOv4 target detection model with an input size of 416mm*416mm to obtain a list of all the Bouding Boxes where the target materials exist in the image, and filter them using the non-maximum suppression (NMS) algorithm to obtain the coordinate position information of the target garbage points that need to be retained;

[0009] Step 3: After the hyperspectral image information is reduced in dimension by the PLS model, the characteristic band information of the target material is obtained and classified;

[0010] Step 4: Traverse the YOLOv4 target detection model to predict the target box confidence, and determine whether the confidence of each target box is greater than 0.8. If the confidence of the target box is greater than 0.8, obtain the carbon value of the corresponding target object according to the target object label information and output the carbon value result;

[0011] Step 5: If the confidence of the target frame is less than 0.8, intercept the area where the hyperspectral camera model assigns the carbon value to each pixel according to the vertex and end point coordinates of the target frame of the target object, calculate the sum of the pixels in the area, and output the final carbon value of the target object.

[0012] Furthermore, the non-maximum suppression NMS algorithm in step 2 is specifically as follows:

[0013]

[0014] Among them, Si represents the score of each border, M represents the box with the highest current score, bi represents a box among the remaining boxes, Nt is the set NMS threshold, and iou is the ratio of the overlapping areas of the two recognition boxes.

[0015] Furthermore, the training process of the YOLOv4 target detection model in step 2 is:

[0016] 2.1. Mark each target material according to its different target materials;

[0017] 2.2. Put the labeled sample data set into the YOLOv4 target detection model according to 416mm*416mm for training.

[0018] Furthermore, CIOU is used to calculate the loss during the training of the YOLOv4 target detection model in 2.2, and the calculation formula is as follows:

[0019] LOSS_CIOU=1-IOU+(ρ 2 *(b,b gt ) / c 2 )+αv

[0020] Where: IOU is the ratio of the intersection and union of the predicted box and the true box, (ρ 2 *(b,b gt ) is the Euclidean distance between the center points of the predicted box and the true box; c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box; the expansion formula of α and the expansion formula of v in the formula are as follows:

[0021] α=v / (1-IOU+v)

[0022] v=(4 / π 2 )*(arctan(w gt / h gt )-arctan(w / h)) 2

[0023] Among them, w represents width; h represents height; gt represents the ground truth data label.

[0024] Furthermore, the step three is specifically as follows:

[0025] 3.1. Use a hyperspectral line scan camera to collect two-dimensional matrix spectral information of the target garbage material;

[0026] 3.2. Use the PLS dimensionality reduction model to reduce the dimension of the acquired two-dimensional matrix spectral information;

[0027] 3.3. Through the PLS dimension reduction model operation, two characteristic target values ​​are extracted as characteristic target data, and all characteristic target data are randomly divided into training set, test set and validation set;

[0028] 3.4. Bring the sample data in the training set into the neural network to complete the hyperspectral data classification training, and use the neural network for the final classification.

[0029] Furthermore, the PLS dimensionality reduction model calculation formula is as follows:

[0030] A=TP T +E (1)

[0031] B=UQ T +F (2)

[0032] Where: A represents the n*m ​​prediction matrix; B represents the n*p response matrix; T and U represent n*l matrices, which are the projections of A and B respectively; P and Q represent the m*l and p*l orthogonal loading matrices; matrices E and F are error terms;

[0033] Formulas (1) and (2) are operated together to complete the dimensionality reduction of the PLS model.

[0034] Furthermore, the hyperspectral data classification training in 3.4 is specifically as follows:

[0035] Bring the data into a hidden layer containing multiple neurons. The mathematical representation of the neuron output is as follows:

[0036]

[0037] Where t is the neuron output, f is the transfer function, b is the bias, is the weight vector, for The transpose of , where vector A is the input matrix vector.

[0038] Optimize neural network parameters using optimizers;

[0039] Use the relu activation function as the activation function;

[0040] The calculation formula of relu function is as follows:

[0041] f(x) = max(0, x)

[0042] Where x is the input value of the function;

[0043] Use the Softmax function as the output layer to achieve target classification;

[0044] The Softmax function is as follows:

[0045]

[0046] Among them, z i is the output value of the ith node, C is the number of output nodes, that is, the number of classification categories, and the meaning of the two e's.

[0047] Furthermore, the step five is specifically as follows:

[0048] 5.1. Collect and summarize the carbon emission coefficients of target materials, and convert the carbon value of each target material according to the carbon emission coefficients of target materials;

[0049] 5.2. Use a hyperspectral line scan camera to collect target materials and obtain pixel information of target materials;

[0050] 5.3. The target material pixel carbon value is calculated based on the target material pixel information and the target material carbon value. The calculation formula is as follows:

[0051]

[0052] 5.4. Repeat steps 5.2 to 5.3, and obtain the average carbon value corresponding to each pixel through comprehensive calculation of a large number of similar target materials;

[0053] 5.5. Real-time sorting to obtain the corresponding target material carbon value results.

[0054] Furthermore, the carbon emission coefficient of the target material in 5.1 is obtained by comprehensively summarizing the carbon emission factor libraries around the world.

[0055] Compared with the prior art, the present invention has the following advantages and effects:

[0056] 1. The present invention can accurately identify the detailed category of each target object through a deep learning target detection method, thereby obtaining the carbon value corresponding to the target object category, and the accuracy of garbage carbon value estimation is high;

[0057] 2. The present invention has a fast calculation speed based on the deep learning target detection method, and can calculate the carbon values ​​of multiple types of targets in a short time;

[0058] 3. The present invention uses a hyperspectral line scan camera. Compared with the three RGB information values ​​of each pixel point of an ordinary industrial 2D camera, each pixel point of a hyperspectral camera contains 200 information values. After classification and conversion, the carbon value of the material calculated based on the pixel points is more accurate;

[0059] 4. The present invention has higher robustness in carbon value estimation by combining the target confidence obtained by deep learning target detection and the carbon value estimated by classification calculation through a hyperspectral line scan camera model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of a method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology of the present invention.

[0061] Figure 2 This is a flow chart of the hyperspectral data model training of the present invention.

[0062] Figure 3 It is a YOLOv4 target detection model training flow chart of the present invention.

[0063] Figure 4 It is a flow chart of calculating the carbon value corresponding to a pixel point according to the present invention. DETAILED DESCRIPTION

[0064] In order to elaborate on the technical scheme adopted by the present invention to achieve the predetermined technical purpose, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only partial embodiments of the present invention, rather than all embodiments, and the technical means or technical features in the embodiments of the present invention can be replaced without paying creative work. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0065] like Figure 1 As shown, a garbage carbon value estimation method based on deep learning and hyperspectral recognition technology of the present invention comprises the following steps:

[0066] Step 1: The 2D industrial camera takes the RGB image of the garbage, and the hyperspectral line scan camera simultaneously collects the hyperspectral image information of the garbage. Hyperspectral recognition technology mainly uses hyperspectral line scan cameras for data collection. The camera mainly collects data in the 900-1700nm band range. This is mainly because the spectra of common materials to be sorted vary greatly within this band, which can effectively identify the target materials. The 2D industrial camera and the hyperspectral line scan camera synchronously collect the target material information of the garbage on the conveyor belt through the encoder.

[0067] Step 2: Input the RGB image into the YOLOv4 target detection model with an input size of 416mm*416mm to obtain a list of all Bouding Boxes where target materials exist in the image, and filter them using the non-maximum suppression (NMS) algorithm to obtain the coordinate position information of the target garbage points that need to be retained.

[0068] The non-maximum suppression NMS algorithm is specifically as follows:

[0069]

[0070] Among them, Si represents the score of each border, M represents the box with the highest current score, bi represents a box among the remaining boxes, Nt is the set NMS threshold, and iou is the ratio of the overlapping areas of the two recognition boxes.

[0071] like Figure 3 As shown in the figure, the training process of the YOLOv4 target detection model is:

[0072] 2.1. Label each target material according to its type and size. For example, a 330mL Nongfu Spring mineral water bottle is labeled, and a 700mL Head & Shoulders shampoo bottle is labeled.

[0073] 2.2. Put the labeled sample data set into the YOLOv4 target detection model according to 416mm*416mm for training.

[0074] CIOU is used to calculate the loss during the training of the YOLOv4 target detection model. The calculation formula is as follows:

[0075] LOSS_CIOU=1-IOU+(ρ 2 *(b,b gt ) / c 2 )+αv

[0076] Where: IOU is the ratio of the intersection and union of the predicted box and the real box, (ρ 2 *(b,b gt) is the Euclidean distance between the center points of the predicted box and the true box; c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box; the expansion formula of α and the expansion formula of v in the formula are as follows:

[0077] α=v / (1-IOU+v)

[0078] v=(4 / π 2 )*(arctan(w gt / h gt )-arctan(w / h)) 2

[0079] Among them, w represents width; h represents height; gt represents the ground truth data label.

[0080] The bbox regression in CIOU can solve the three problems of coverage area, center point distance and aspect ratio in the target material identification process, and greatly improve the ability to detect target waste. After training in the YOLOv4 target detection model, the model recognition accuracy can be effectively improved, and the accuracy of the target material carbon value estimation can be increased in the final carbon value analysis.

[0081] Step 3: After the hyperspectral image information is reduced in dimension by the PLS model, the characteristic band information of the target material is obtained and classified.

[0082] The PLS dimensionality reduction model is the partial least squares regression classification method. Since there are usually more than 200 hyperspectral band data, which contain a lot of redundant information, before the next step of classification, it is necessary to select bands or features that contain large amounts of information to reduce the redundancy of the data, project the original high-dimensional data into a low-dimensional space, and obtain more streamlined original data, which can effectively reduce the amount of calculation and improve the model operation speed.

[0083] The calculation formula of the PLS dimensionality reduction model is as follows:

[0084] A=TP T +E (1)

[0085] B=UQ T +F (2)

[0086] Where: A represents the n*m ​​prediction matrix; B represents the n*p response matrix; T and U represent n*l matrices, which are the projections of A and B respectively; P and Q represent the m*l and p*l orthogonal loading matrices; matrices E and F are error terms;

[0087] Formulas (1) and (2) are operated together to complete the dimensionality reduction of the PLS model.

[0088] like Figure 2 As shown in the figure, the hyperspectral data model training is specifically as follows:

[0089] 3.1. Use a hyperspectral line scan camera to collect two-dimensional matrix spectral information of the target garbage material;

[0090] 3.2. Use the PLS dimensionality reduction model to reduce the dimension of the acquired two-dimensional matrix spectral information;

[0091] 3.3. Through the PLS dimension reduction model operation, two characteristic target values ​​are extracted as characteristic target data, and all characteristic target data are randomly divided into training set, test set and validation set;

[0092] 3.4. Bring the sample data in the training set into the neural network to complete the hyperspectral data classification training, and use the neural network for the final classification. This can achieve a better material classification effect, so that the carbon value of the pixel point can be accurately recorded, and the final calculation accuracy of the carbon value of the garbage pixel point can be improved.

[0093] Hyperspectral data classification training is specifically as follows:

[0094] Bring the data into a hidden layer containing multiple neurons. The mathematical representation of the neuron output is as follows:

[0095]

[0096] Where t is the neuron output, f is the transfer function, b is the bias, is the weight vector, for The transpose of , where vector A is the input matrix vector.

[0097] Optimize neural network parameters using optimizers;

[0098] Use the relu activation function as the activation function;

[0099] The calculation formula of relu function is as follows:

[0100] f(x) = max(0, x)

[0101] Where x is the input value of the function;

[0102] Use the Softmax function as the output layer to achieve target classification;

[0103] The Softmax function is as follows:

[0104]

[0105] Among them, z iis the output value of the ith node, C is the number of output nodes, that is, the number of classification categories, and the meaning of the two e's.

[0106] Step 4: Traverse the YOLOv4 target detection model to predict the target box confidence, and determine whether the confidence of each target box is greater than 0.8. If the confidence of the target box is greater than 0.8, obtain the carbon value of the corresponding target object according to the target object label information and output the carbon value result;

[0107] Step 5: If the confidence of the target frame is less than 0.8, intercept the area where the hyperspectral camera model assigns the carbon value to each pixel according to the vertex and end point coordinates of the target frame of the target object, calculate the sum of the pixels in the area, and output the final carbon value of the target object.

[0108] like Figure 4 As shown in the figure, the method for estimating the carbon value of the target material is as follows:

[0109] 5.1. Collect and summarize the carbon emission coefficients of the target materials, and convert the carbon value of each target material according to the carbon emission coefficients of the target materials; the carbon emission coefficients of the target materials come from carbon emission factor libraries around the world, such as the carbon emission factor library of the Department for Environment, Food and Rural Affairs (DEFRA) of the United Kingdom and the carbon emission factor library of France, and the carbon emission coefficients of each target material are obtained by comprehensive summary.

[0110] 5.2. Use a hyperspectral line scan camera to collect target materials and obtain pixel information of target materials;

[0111] 5.3. The target material pixel carbon value is calculated based on the target material pixel information and the target material carbon value. The calculation formula is as follows:

[0112]

[0113] 5.4. Repeat steps 5.2 to 5.3, and obtain the average carbon value corresponding to each pixel through comprehensive calculation of a large number of similar target materials;

[0114] 5.5. Real-time sorting to obtain the corresponding target material carbon value results.

[0115] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A garbage carbon value estimation method based on deep learning and hyperspectral recognition technology, characterized in that The following steps are involved: Step 1: The 2D industrial camera takes RGB images of the garbage, and the hyperspectral line scan camera simultaneously collects the hyperspectral image information of the garbage; Step 2: Input the RGB image into the YOLOv4 target detection model with an input size of 416mm*416mm to obtain a list of all the Bouding Boxes containing the target materials in the image, and filter them using the non-maximum suppression (NMS) algorithm to obtain the coordinate position information of the target garbage points that need to be retained. Step 3: After the hyperspectral image information is reduced in dimension by the PLS model, the characteristic band information of the target material is obtained and classified; Step 4: Traverse the YOLOv4 target detection model to predict the target box confidence, and determine whether the confidence of each target box is greater than 0.

8. If the confidence of the target box is greater than 0.8, obtain the carbon value of the corresponding target object according to the target object label information and output the carbon value result; Step 5: If the confidence of the target frame is less than 0.8, intercept the area where the hyperspectral camera model assigns the carbon value to each pixel according to the vertex and end point coordinates of the target frame of the target object, calculate the sum of the pixels in the area, and output the final carbon value of the target object.

2. According to claim 1, a method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology is characterized by: The non-maximum suppression NMS algorithm in step 2 is specifically as follows: Among them, Si represents the score of each box, M represents the box with the highest current score, bi represents a box of the remaining boxes, Nt is the set NMS threshold, and IOU is the ratio of the intersection and union of the predicted box and the true box.

3. The method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology according to claim 1 is characterized in that: The training process of the YOLOv4 target detection model in step 2 is: 2.

1. Mark each target material according to its different target materials; 2.

2. Put the labeled sample data set into the YOLOv4 target detection model according to 416mm*416mm for training.

4. The method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology according to claim 3 is characterized by: In the YOLOv4 target detection model training process described in 2.2, CIOU is used to calculate the loss, and the calculation formula is as follows: LOSS_CIOU=1-IOUU+(ρ 2 *(b,b gt ) / c 2 )+αv Where: IOU is the ratio of the intersection and union of the predicted box and the true box, (ρ 2 *(b,b gt ) is the Euclidean distance between the center points of the predicted box and the true box; c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box; the expansion formula of α and the expansion formula of v in the formula are as follows: α=v / (1-IOU+v) v=(4 / π 2 )*(arctan(w gt / h gt )-arctan(w / h)) 2 Among them, w represents width; h represents height; gt represents the ground truth data label.

5. The method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology according to claim 1 is characterized by: The step three is specifically as follows: 3.

1. Use a hyperspectral line scan camera to collect two-dimensional matrix spectral information of the target garbage material; 3.

2. Use the PLS dimensionality reduction model to reduce the dimension of the acquired two-dimensional matrix spectral information; 3.

3. Through the PLS dimension reduction model operation, two characteristic target values ​​are extracted as characteristic target data, and all characteristic target data are randomly divided into training set, test set and validation set; 3.

4. Bring the sample data in the training set into the neural network to complete the hyperspectral data classification training, and use the neural network for the final classification.

6. The method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology according to claim 5 is characterized by: The PLS dimensionality reduction model calculation formula is as follows: A=TP T +E (1) B=UQ T +F (2) Where: A represents the n*m ​​prediction matrix; B represents the n*p response matrix; T and U represent n*l matrices, which are the projections of A and B respectively; P and Q represent the m*l and p*l orthogonal loading matrices; matrices E and F are error terms; Formulas (1) and (2) are operated together to complete the dimensionality reduction of the PLS model.

7. The method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology according to claim 5 is characterized by: The hyperspectral data classification training in 3.4 is specifically as follows: Bring the data into a hidden layer containing multiple neurons. The mathematical representation of the neuron output is as follows: Where t is the neuron output, f is the transfer function, b is the bias, is the weight vector, for The transpose of , vector A is the input matrix vector; Optimize neural network parameters using optimizers; Use the relu activation function as the activation function; The calculation formula of relu function is as follows: f(x)=max(0,x) Where x is the input value of the function; Use the Softmax function as the output layer to achieve target classification; The Softmax function is as follows: Among them, z i is the output value of the ith node, C is the number of output nodes, that is, the number of classification categories, and e is a natural constant.

8. The method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology according to claim 1 is characterized by: The step five is specifically as follows: 5.

1. Collect and summarize the carbon emission coefficients of target materials, and convert the carbon value of each target material according to the carbon emission coefficients of target materials; 5.

2. Use a hyperspectral line scan camera to collect target materials and obtain pixel information of target materials; 5.

3. The target material pixel carbon value is calculated based on the target material pixel information and the target material carbon value. The calculation formula is as follows: 5.

4. Repeat steps 5.2 to 5.3, and obtain the average carbon value corresponding to each pixel through comprehensive calculation of a large number of similar target materials; 5.

5. Real-time sorting to obtain the corresponding target material carbon value results.

9. The method for estimating the carbon value of garbage based on deep learning and hyperspectral recognition technology according to claim 8 is characterized by: The carbon emission coefficients of the target materials in 5.1 are obtained by comprehensively summarizing carbon emission factor libraries from all over the world.

Images