A method for garbage category detection using neural network at a fixed distance
By using a deep learning-based SSD object detection neural network at a fixed distance, the garbage pictures are grouped and processed, and the problem of low garbage category detection speed and accuracy in the prior art is solved, and efficient garbage category detection is achieved.
Patent Information
- Application Number
- CN202111192061.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-13
- Publication Date
- 2025-01-10
- Estimated Expiration
- 2041-10-13
AI Technical Summary
The existing SSD object detection neural network based on deep learning is not very detection speed and accuracy when facing a large number of dense garbage distributed images, and cannot effectively process a large amount of garbage with different shapes and sizes.
By using the SSD object detection neural network based on deep learning at a fixed distance, setting a fixed garbage disposal point and camera device location, adjusting the network's processing flow, dividing the pictures into two groups: large target and small target, and using detection classification layer to perform Screening and processing greatly simplify the classification judgment process.
It significantly improves the speed and accuracy of garbage category detection, and realizes efficient garbage category detection at fixed distances.
Abstract
Description
Technical Field
[0001] The present invention relates to a method for detecting garbage categories by using a neural network, and in particular to a method for detecting garbage categories by using a neural network at a fixed distance. Background Art
[0002] As people's living standards improve, more and more garbage is generated. At present, these garbage are mainly divided into recyclable garbage, hazardous garbage, kitchen waste and other garbage. By classifying and managing garbage, garbage resources can be recycled to the maximum extent, while reducing the amount of garbage disposal.
[0003] At present, neural networks are usually used for garbage classification detection. The accuracy and real-time performance of the neural network model directly determine the accuracy and real-time performance of garbage classification detection. The SSD target detection neural network based on deep learning is a commonly used neural network for garbage classification.
[0004] The existing SSD target detection neural network based on deep learning usually includes the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer, the fifth convolution layer, the sixth convolution layer, the seventh convolution layer, the eighth convolution layer, the ninth convolution layer and the detection classification layer arranged in sequence. The first convolution layer contains two [3,3,64] convolution networks and a 2X2 maximum pooling layer for outputting images of size (150,150,64). The second convolution layer contains two [3,3,128] convolution networks and a 2X2 maximum pooling layer for outputting images of size (75,75,128). The third convolution layer contains three [ 3,3,256] convolutional network and a 2X2 maximum pooling layer, used to output a picture of size (38,38,256). The fourth convolutional layer is the feature extraction layer, which contains three [3,3,512] convolutional networks and a 2X2 maximum pooling layer, which is used to generate a feature picture of size 38*38. The fifth convolutional layer is the feature extraction layer, which contains three [3,3,1024] convolutional networks, a 2X2 maximum pooling layer and two [3,3] convolutional networks, which is used to generate a feature picture of size 19*19. The sixth convolutional layer is the feature extraction layer, which contains a [1,1,512] convolutional network and a [3,3] convolutional network with a step size of 2. The seventh convolution layer is a feature extraction layer, which includes a [1,1,256] convolution network and a [3,3] convolution network with a step size of 2, which is used to generate a feature image of size 5*5. The eighth convolution layer is a feature extraction layer, which includes a [1,1,256] convolution network and a [3,3,256] convolution network with valid padding, which is used to generate a feature image of size 3*3. The ninth convolution layer is a feature extraction layer, which includes a [1,1,256] convolution network and a [3,3,256] convolution network with valid padding, which is used to generate a feature image of size 3*3. The detection and classification layer is used to obtain feature images from all feature extraction layers and generate a preset default candidate frame for each pixel in each feature image. Then, the detection function is used to predict the category and position of the default candidate frame of each pixel in each feature image to obtain the prediction result, which includes the score of each garbage category of each default candidate frame and the offset value of each default candidate frame relative to the target frame. Finally, according to the score of each garbage category of each default candidate frame and the offset value of each default candidate frame relative to the target frame, the redundant target frames are eliminated by the non-maximum suppression method.The remaining target boxes are marked on a picture with a pixel size of 300*300 to obtain the detection results, where each pixel in the feature picture with a size of 38*38 generates 4 default candidate boxes, each pixel in the feature picture with a size of 19*19 generates 6 default candidate boxes, each pixel in the feature picture with a size of 10*10 generates 6 default candidate boxes, each pixel in the feature picture with a size of 5*5 generates 6 default candidate boxes, each pixel in the feature picture with a size of 3*3 generates 4 default candidate boxes, and each pixel in the feature picture with a size of 1*1 generates 4 default candidate boxes. ,
[0005] The process of using the above-mentioned SSD target detection neural network based on deep learning to classify and detect garbage is as follows: the detection image is resized to a pixel size of 300*300 through the resize function and then input into the first convolution layer. The first convolution layer processes the input image and outputs an image of size (150,150,64) to the second convolution layer. The second convolution layer processes the input image and outputs an image of size (75,75,128) to the third convolution layer. The third convolution layer inputs the image After processing, the output image of size (38,38,256) is sent to the fourth convolutional layer. The fourth convolutional layer processes the input image and outputs an image of size (38,38,512) to the fifth convolutional layer and the detection and classification layer. The fifth convolutional layer processes the input image and outputs an image of size (19,19,1024) to the sixth convolutional layer and the detection and classification layer. The sixth convolutional layer processes the input image and outputs an image of size (10,10,512) to the seventh convolutional layer and the detection and classification layer. The seven convolutional layers process the input images and output images of size (5,5,256) to the eighth convolutional layer and the detection and classification layer. The eighth convolutional layer processes the input images and outputs images of size (3,3,256) to the ninth convolutional layer and the detection and classification layer. The ninth convolutional layer processes the input images and outputs images of size (1,1,256) to the detection and classification layer. The detection and classification layer generates a preset default candidate box for each pixel in each image input, and then predicts the garbage attribution category and position of the default candidate box of each pixel in each image through the detection function to obtain the prediction result. The prediction result includes the score of each garbage attribution category of each default candidate box and the offset value of each default candidate box relative to the target box. Finally, according to the obtained scores of each garbage attribution category of each default candidate box and the offset value of each default candidate box relative to the target box, the redundant target boxes are eliminated by the non-maximum suppression method, and the remaining target boxes are marked on the picture with a pixel size of 300*300 to obtain the detection result.
[0006] However, when faced with a large number of images with densely distributed garbage, the above-mentioned SSD object detection neural network based on deep learning has to process the garbage one by one in the images because there is too much garbage in the images and the garbage is of different shapes and sizes, resulting in low speed and accuracy in garbage category detection. Summary of the invention
[0007] The technical problem to be solved by the present invention is to provide a method for detecting garbage categories by using a neural network at a fixed distance with high detection speed and detection accuracy.
[0008] The technical solution adopted by the present invention to solve the above technical problems is: a method for garbage category detection using a neural network at a fixed distance, wherein the neural network is an SSD target detection neural network based on deep learning, and the SSD target detection neural network based on deep learning comprises a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a seventh convolution layer, an eighth convolution layer, a ninth convolution layer and a detection classification layer arranged in sequence, wherein the first convolution layer comprises two [ 3,3,64] convolutional network and a 2X2 maximum pooling layer, used to output images of size (150,150,64). The second convolutional layer contains two [3,3,128] convolutional networks and a 2X2 maximum pooling layer, used to output images of size (75,75,128). The third convolutional layer contains three [3,3,256] convolutional networks and a 2X2 maximum pooling layer, used to output images of size (38,38,256). The fourth convolutional layer is a feature extraction layer, which contains three [3,3,5 12] convolutional network and a 2X2 maximum pooling layer, used to generate a feature image of size 38*38. The fifth convolutional layer is a feature extraction layer, which includes three [3,3, 1024] convolutional networks, a 2X2 maximum pooling layer and two [3,3] convolutional networks, used to generate a feature image of size 19*19. The sixth convolutional layer is a feature extraction layer, which includes a [1,1, 512] convolutional network and a [3,3] convolutional network with a step size of 2, used to generate a feature image of size 10*10. The seventh convolutional layer is The feature extraction layer includes a [1,1, 256] convolutional network and a [3,3] convolutional network with a step size of 2, which is used to generate a feature image of size 5*5. The eighth convolutional layer is the feature extraction layer, which includes a [1,1, 256] convolutional network and a [3,3, 256] convolutional network with valid padding, which is used to generate a feature image of size 3*3. The ninth convolutional layer is the feature extraction layer, which includes a [1,1, 256] convolutional network and a [3,3,256] convolutional network, used to generate a feature image of size 1*1, the detection and classification layer can receive the feature images provided by all feature extraction layers, and generate a pre-set default candidate box for each pixel in each feature image, and then use the detection function to predict the garbage attribution category and position of the default candidate box of each pixel in each feature image to obtain the prediction result, which includes the score of each garbage attribution category of each default candidate box and the offset value of each default candidate box relative to its target box. Finally, according to the obtained score of each garbage attribution category of each default candidate box and The offset value of each default candidate frame coordinate relative to its target frame coordinate is used to obtain the detection result. In this case, each pixel in the feature image of size 38*38 generates 4 default candidate frames, each pixel in the feature image of size 19*19 generates 6 default candidate frames, each pixel in the feature image of size 10*10 generates 6 default candidate frames, each pixel in the feature image of size 5*5 generates 6 default candidate frames, each pixel in the feature image of size 3*3 generates 4 default candidate frames, and each pixel in the feature image of size 1*1 generates 4 default candidate frames. The specific detection steps are:
[0009] (1) The detection image is resized to a pixel size of 300*300 through the resize function and then input into the first convolutional layer. The two-dimensional coordinates of the image with a pixel size of 300*300 take its lower left corner as the origin, and the directions of the two edges intersecting the lower left corner as the horizontal and vertical coordinates respectively;
[0010] (2) The first convolutional layer processes the input image and outputs an image of size (150, 150, 64) to the second convolutional layer;
[0011] (3) The second convolutional layer processes the input image and outputs an image of size (75, 75, 128) to the third convolutional layer;
[0012] (4) The third convolutional layer processes the input image and outputs an image of size (38,38,256) to the fourth convolutional layer;
[0013] (5) The fourth convolutional layer processes the input image and outputs an image of size (38,38,512) to the fifth convolutional layer and the detection and classification layer;
[0014] (6) The fifth convolutional layer processes the input image and outputs an image of size (19, 19, 1024) to the sixth convolutional layer and the detection and classification layer;
[0015] (7) The sixth convolutional layer processes the input image and outputs an image of size (10,10,512) to the seventh convolutional layer and the detection and classification layer;
[0016] (8) The seventh convolutional layer processes the input image and outputs an image of size (5,5,256) to the eighth convolutional layer and the detection and classification layer;
[0017] (9) The eighth convolutional layer processes the input image and outputs an image of size (3,3,256) to the ninth convolutional layer and the detection and classification layer;
[0018] (10) The ninth convolutional layer processes the input image and outputs an image of size (1,1,256) to the detection and classification layer;
[0019] (11) Use the detection classification layer to perform classification processing. The specific steps are as follows:
[0020] S1. The detection and classification layer generates a preset default candidate box for each pixel in each picture input therein, and then predicts the garbage category and position of the default candidate box of each pixel in each picture through the detection function to obtain the prediction result, which includes the score of each garbage category of each default candidate box and the offset value of each default candidate box coordinate relative to the target box coordinate, where the two-dimensional coordinate of each picture is respectively taken as the origin at its lower left corner, and the directions of the two edges intersecting the lower left corner are respectively taken as the horizontal coordinate and the vertical coordinate;
[0021] S2, adding the coordinates of each default candidate box in each picture to the offset value of the coordinates of the default candidate box relative to the coordinates of the target box to obtain the coordinate value of the target box of the default candidate box, and adjusting the position of the default candidate box so that its position overlaps with the position of the target box. At this time, the default candidate box is the target box;
[0022] S3, counting the scores of all garbage categories at each default candidate frame in each image, and taking the garbage category with the highest score as the label category of the default candidate frame, which is also the label category of the target frame corresponding to the default candidate frame;
[0023] S4. Process each image as follows: classify all target frames according to label categories, divide target frames belonging to the same label category into one category, then calculate the area of each target frame and sort them in ascending order, delete the smallest 10% and the largest 10% of all target frames, and round up if 10% is not an integer;
[0024] S5, calculating the area of the target box in each image obtained in step S4, taking the target box with an area less than 10% of 300*300 as a small target, and taking the target box with an area greater than or equal to 10% of 300*300 as a large target;
[0025] S6. The six images obtained in step S4 are processed as follows: the scores of the label categories of the large targets in the three images with image sizes of 1*1, 3*3, and 5*5 are set to zero, and the scores of the label categories of the small targets in the three images with image sizes of 38*38, 19*19, and 10*10 are set to zero, and the target frames with non-zero label category scores in each image are screened out;
[0026] S7, the target frames selected in each image are processed as follows: the label categories of the selected target frames are counted, the target frames belonging to the same label category are regarded as one category, the target frame with the highest label category score in the category is regarded as the reference frame, and the other target frames are regarded as the candidate frames, and each candidate frame is compared with the reference frame as follows: if the ratio of the overlapping area of the candidate frame and the reference frame to the sum of the areas of the candidate frame and the reference frame is greater than 0.5, the candidate frame is removed, thereby obtaining the remaining target frames in each image;
[0027] S8. Obtain the coordinates of the remaining target frames in each image obtained in step S7, multiply the coordinates by the compression factor of the image relative to the image with a pixel size of 300*300 in step (1), and obtain the actual coordinates of the target frame. Then, annotate the target frame in the image with a pixel size of 300*300 according to the actual coordinates of the target frame, thereby obtaining an annotated image with a pixel size of 300*300. The marking of the garbage location and category is completed, and the garbage classification detection is completed.
[0028] The SSD target detection neural network based on deep learning is pre-trained by the following method: first, garbage pictures on the assembly line in the garbage treatment plant are collected; then the collected garbage pictures are labeled with garbage categories and locations through labeling software, and these labeled garbage pictures are used as data sets to train the SSD target detection neural network based on deep learning.
[0029] Compared with the prior art, the advantage of the present invention is that when the classification processing is performed by the SSD target detection neural network based on deep learning, since the garbage disposal points are fixed, the position of the camera device for taking garbage pictures set at each disposal point is also fixed, so the classification detection of garbage always maintains a fixed distance, and the size of the corresponding garbage also has a certain range in the taken garbage pictures. If it exceeds or is less than this range, the SSD target detection neural network based on deep learning does not need to make further judgments on it. Therefore, when the detection and classification layer detects and judges, after determining the target frame, the target frame is screened, and the screened target frame is classified into large targets and small targets, and then the 6 pictures are divided into two groups of large targets and small targets for separate processing, which greatly simplifies the classification and judgment process of the SSD target detection neural network based on deep learning, and effectively improves the speed and accuracy of the SSD target detection neural network based on deep learning. Therefore, the method of the present invention has high detection speed and detection accuracy. DETAILED DESCRIPTION
[0030] The present invention is further described in detail below with reference to the embodiments.
[0031] Embodiment: A method for detecting garbage categories using a neural network at a fixed distance, wherein the neural network is an SSD target detection neural network based on deep learning, and the SSD target detection neural network based on deep learning comprises a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a seventh convolution layer, an eighth convolution layer, a ninth convolution layer and a detection classification layer arranged in sequence, wherein the first convolution layer comprises two [3,3,64] convolution networks and a 2X2 The maximum pooling layer is used to output images of size (150, 150, 64). The second convolutional layer contains two [3, 3, 128] convolutional networks and a 2X2 maximum pooling layer, which is used to output images of size (75, 75, 128). The third convolutional layer contains three [3, 3, 256] convolutional networks and a 2X2 maximum pooling layer, which is used to output images of size (38, 38, 256). The fourth convolutional layer is a feature extraction layer, which contains three [3, 3, 512] convolutional networks and a 2X 2 maximum pooling layers, used to generate feature images of size 38*38, the fifth convolution layer is the feature extraction layer, which contains three [3,3, 1024] convolution networks, a 2X2 maximum pooling layer and two [3,3] convolution networks, used to generate feature images of size 19*19, the sixth convolution layer is the feature extraction layer, which contains a [1,1, 512] convolution network and a [3,3] convolution network with a step size of 2, used to generate feature images of size 10*10, the seventh convolution layer is the feature extraction layer, It contains a [1,1,256] convolutional network and a [3,3] convolutional network with a step size of 2, which are used to generate feature images of size 5*5. The eighth convolutional layer is the feature extraction layer, which contains a [1,1,256] convolutional network and a [3,3,256] convolutional network with valid padding, which are used to generate feature images of size 3*3. The ninth convolutional layer is the feature extraction layer, which contains a [1,1,256] convolutional network and a [3,3,256] convolutional network, used to generate a feature image of size 1*1, the detection and classification layer can receive the feature images provided by all feature extraction layers, and generate a pre-set default candidate box for each pixel in each feature image, and then use the detection function to predict the garbage attribution category and position of the default candidate box of each pixel in each feature image to obtain the prediction result, which includes the score of each garbage attribution category of each default candidate box and the offset value of each default candidate box relative to its target box. Finally, according to the obtained score of each garbage attribution category of each default candidate box and each The offset value of the default candidate frame coordinates relative to the target frame coordinates is used to obtain the detection result, wherein each pixel in the feature image of size 38*38 generates 4 default candidate frames, each pixel in the feature image of size 19*19 generates 6 default candidate frames, each pixel in the feature image of size 10*10 generates 6 default candidate frames, each pixel in the feature image of size 5*5 generates 6 default candidate frames, each pixel in the feature image of size 3*3 generates 4 default candidate frames, and each pixel in the feature image of size 1*1 generates 4 default candidate frames. The specific detection steps are:,
[0032] (1) The detection image is resized to a pixel size of 300*300 through the resize function and then input into the first convolutional layer. The two-dimensional coordinates of the image with a pixel size of 300*300 take its lower left corner as the origin, and the directions of the two edges intersecting the lower left corner as the horizontal and vertical coordinates respectively;
[0033] (2) The first convolutional layer processes the input image and outputs an image of size (150, 150, 64) to the second convolutional layer;
[0034] (3) The second convolutional layer processes the input image and outputs an image of size (75, 75, 128) to the third convolutional layer;
[0035] (4) The third convolutional layer processes the input image and outputs an image of size (38,38,256) to the fourth convolutional layer;
[0036] (5) The fourth convolutional layer processes the input image and outputs an image of size (38,38,512) to the fifth convolutional layer and the detection and classification layer;
[0037] (6) The fifth convolutional layer processes the input image and outputs an image of size (19, 19, 1024) to the sixth convolutional layer and the detection and classification layer;
[0038] (7) The sixth convolutional layer processes the input image and outputs an image of size (10,10,512) to the seventh convolutional layer and the detection and classification layer;
[0039] (8) The seventh convolutional layer processes the input image and outputs an image of size (5,5,256) to the eighth convolutional layer and the detection and classification layer;
[0040] (9) The eighth convolutional layer processes the input image and outputs an image of size (3,3,256) to the ninth convolutional layer and the detection and classification layer;
[0041] (10) The ninth convolutional layer processes the input image and outputs an image of size (1,1,256) to the detection and classification layer;
[0042] (11) Use the detection classification layer to perform classification processing. The specific steps are as follows:
[0043] S1. The detection and classification layer generates a preset default candidate box for each pixel in each picture input therein, and then predicts the garbage category and position of the default candidate box of each pixel in each picture through the detection function to obtain the prediction result, which includes the score of each garbage category of each default candidate box and the offset value of each default candidate box coordinate relative to the target box coordinate, where the two-dimensional coordinate of each picture is respectively taken as the origin at its lower left corner, and the directions of the two edges intersecting the lower left corner are respectively taken as the horizontal coordinate and the vertical coordinate;
[0044] S2, adding the coordinates of each default candidate box in each picture to the offset value of the coordinates of the default candidate box relative to the coordinates of the target box to obtain the coordinate value of the target box of the default candidate box, and adjusting the position of the default candidate box so that its position overlaps with the position of the target box. At this time, the default candidate box is the target box;
[0045] S3, counting the scores of all garbage categories at each default candidate frame in each image, and taking the garbage category with the highest score as the label category of the default candidate frame, which is also the label category of the target frame corresponding to the default candidate frame;
[0046] S4. Process each image as follows: classify all target frames according to label categories, divide target frames belonging to the same label category into one category, then calculate the area of each target frame and sort them in ascending order, delete the smallest 10% and the largest 10% of all target frames, and round up if 10% is not an integer;
[0047] S5, calculating the area of the target box in each image obtained in step S4, taking the target box with an area less than 10% of 300*300 as a small target, and taking the target box with an area greater than or equal to 10% of 300*300 as a large target;
[0048] S6. The six images obtained in step S4 are processed as follows: the scores of the label categories of the large targets in the three images with image sizes of 1*1, 3*3, and 5*5 are set to zero, and the scores of the label categories of the small targets in the three images with image sizes of 38*38, 19*19, and 10*10 are set to zero, and the target frames with non-zero label category scores in each image are screened out;
[0049] S7, the target frames selected in each image are processed as follows: the label categories of the selected target frames are counted, the target frames belonging to the same label category are regarded as one category, the target frame with the highest label category score in the category is regarded as the reference frame, and the other target frames are regarded as the candidate frames, and each candidate frame is compared with the reference frame as follows: if the ratio of the overlapping area of the candidate frame and the reference frame to the sum of the areas of the candidate frame and the reference frame is greater than 0.5, the candidate frame is removed, thereby obtaining the remaining target frames in each image;
[0050] S8. Obtain the coordinates of the remaining target frames in each image obtained in step S7, multiply the coordinates by the compression factor of the image relative to the image with a pixel size of 300*300 in step (1), and obtain the actual coordinates of the target frame. Then, annotate the target frame in the image with a pixel size of 300*300 according to the actual coordinates of the target frame, thereby obtaining an annotated image with a pixel size of 300*300. The marking of the garbage location and category is completed, and the garbage classification detection is completed.
[0051] In this embodiment, the SSD target detection neural network based on deep learning is pre-trained using the following method: first, garbage pictures on the assembly line in the garbage treatment plant are collected; then the collected garbage pictures are labeled with garbage categories and locations through labeling software, and these labeled garbage pictures are used as data sets to train the SSD target detection neural network based on deep learning.
Claims
1. A method for garbage category detection using a neural network at a fixed distance, wherein the neural network is an SSD target detection neural network based on deep learning, and the SSD target detection neural network based on deep learning comprises a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a seventh convolution layer, an eighth convolution layer, a ninth convolution layer and a detection classification layer arranged in sequence, wherein the first convolution layer comprises two [3,3,64] convolution networks and a 2X2 maximum pooling layer for outputting a picture of size (150,150,64), and the second convolution layer comprises two [3,3,128] convolution networks and a 2X2 The maximum pooling layer is used to output a picture of size (75,75,128). The third convolution layer contains three [3,3,256] convolution networks and a 2X2 maximum pooling layer, which is used to output a picture of size (38,38,256). The fourth convolution layer is a feature extraction layer, which contains three [3,3,512] convolution networks and a 2X2 maximum pooling layer, which is used to generate a feature picture of size 38*38. The fifth convolution layer is a feature extraction layer, which contains three [3,3,1024] convolution networks, a 2X2 maximum pooling layer and two [3,3] convolution networks, which is used to generate a feature picture of size 19*19. The sixth convolution layer is a feature extraction layer, which contains a [1,1 , 512] convolutional network and a [3,3] convolutional network with a step size of 2, used to generate a feature image of size 10*10. The seventh convolutional layer is the feature extraction layer, which includes a [1,1, 256] convolutional network and a [3,3] convolutional network with a step size of 2, used to generate a feature image of size 5*5. The eighth convolutional layer is the feature extraction layer, which includes a [1,1, 256] convolutional network and a [3,3, 256] convolutional network with valid padding, used to generate a feature image of size 3*3. The ninth convolutional layer is the feature extraction layer, which includes a [1,1, 256] convolutional network and a [3,3, 2 56] convolutional network, used to generate a feature image of size 1*1, the detection classification layer can receive the feature images provided by all feature extraction layers, and generate a pre-set default candidate frame for each pixel in each feature image, and then predict the garbage belonging category and position of the default candidate frame of each pixel in each feature image through the detection function to obtain a prediction result, which includes the score of each garbage belonging category of each default candidate frame and the offset value of each default candidate frame relative to its target frame. Finally, according to the obtained score of each garbage belonging category of each default candidate frame and the offset value of each default candidate frame coordinate relative to its target frame coordinate, the detection result is obtained, wherein, Each pixel in the feature image of size 38*38 generates 4 default candidate frames, each pixel in the feature image of size 19*19 generates 6 default candidate frames, each pixel in the feature image of size 10*10 generates 6 default candidate frames, each pixel in the feature image of size 5*5 generates 6 default candidate frames, each pixel in the feature image of size 3*3 generates 4 default candidate frames, and each pixel in the feature image of size 1*1 generates 4 default candidate frames. The specific detection steps are: (1) The detection image is resized to a pixel size of 300*300 through the resize function and then input into the first convolutional layer. The two-dimensional coordinates of the image with a pixel size of 300*300 take its lower left corner as the origin, and the directions of the two edges intersecting the lower left corner as the horizontal and vertical coordinates respectively; (2) The first convolutional layer processes the input image and outputs an image of size (150, 150, 64) to the second convolutional layer; (3) The second convolutional layer processes the input image and outputs an image of size (75, 75, 128) to the third convolutional layer; (4) The third convolutional layer processes the input image and outputs an image of size (38,38,256) to the fourth convolutional layer; (5) The fourth convolutional layer processes the input image and outputs an image of size (38,38,512) to the fifth convolutional layer and the detection and classification layer; (6) The fifth convolutional layer processes the input image and outputs an image of size (19, 19, 1024) to the sixth convolutional layer and the detection and classification layer; (7) The sixth convolutional layer processes the input image and outputs an image of size (10,10,512) to the seventh convolutional layer and the detection and classification layer; (8) The seventh convolutional layer processes the input image and outputs an image of size (5,5,256) to the eighth convolutional layer and the detection and classification layer; (9) The eighth convolutional layer processes the input image and outputs an image of size (3,3,256) to the ninth convolutional layer and the detection and classification layer; (10) The ninth convolutional layer processes the input image and outputs an image of size (1,1,256) to the detection and classification layer; (11) Use the detection classification layer to perform classification processing. The specific steps are as follows: S1. The detection and classification layer generates a preset default candidate box for each pixel in each picture input therein, and then predicts the garbage category and position of the default candidate box of each pixel in each picture through the detection function to obtain the prediction result, which includes the score of each garbage category of each default candidate box and the offset value of each default candidate box coordinate relative to the target box coordinate, where the two-dimensional coordinate of each picture is respectively taken as the origin at its lower left corner, and the directions of the two edges intersecting the lower left corner are respectively taken as the horizontal coordinate and the vertical coordinate; S2, adding the coordinates of each default candidate box in each picture to the offset value of the coordinates of the default candidate box relative to the coordinates of the target box to obtain the coordinate value of the target box of the default candidate box, and adjusting the position of the default candidate box so that its position overlaps with the position of the target box. At this time, the default candidate box is the target box; S3, counting the scores of all garbage categories at each default candidate frame in each image, and taking the garbage category with the highest score as the label category of the default candidate frame, which is also the label category of the target frame corresponding to the default candidate frame; S4. Process each image as follows: classify all target frames according to label categories, divide target frames belonging to the same label category into one category, then calculate the area of each target frame and sort them in ascending order, delete the smallest 10% and the largest 10% of all target frames, and round up if 10% is not an integer; S5, calculating the area of the target box in each image obtained in step S4, taking the target box with an area less than 10% of 300*300 as a small target, and taking the target box with an area greater than or equal to 10% of 300*300 as a large target; S6. The six images obtained in step S4 are processed as follows: the scores of the label categories of the large targets in the three images with image sizes of 1*1, 3*3, and 5*5 are set to zero, and the scores of the label categories of the small targets in the three images with image sizes of 38*38, 19*19, and 10*10 are set to zero, and the target frames with non-zero label category scores in each image are screened out; S7, the target frames selected in each image are processed as follows: the label categories of the selected target frames are counted, the target frames belonging to the same label category are regarded as one category, the target frame with the highest label category score in the category is regarded as the reference frame, and the other target frames are regarded as the candidate frames, and each candidate frame is compared with the reference frame as follows: if the ratio of the overlapping area of the candidate frame and the reference frame to the sum of the areas of the candidate frame and the reference frame is greater than 0.5, the candidate frame is removed, thereby obtaining the remaining target frames in each image; S8. Obtain the coordinates of the remaining target frames in each image obtained in step S7, multiply the coordinates by the compression factor of the image relative to the image with a pixel size of 300*300 in step (1), and obtain the actual coordinates of the target frame. Then, annotate the target frame in the image with a pixel size of 300*300 according to the actual coordinates of the target frame, thereby obtaining an annotated image with a pixel size of 300*300. The marking of the garbage location and category is completed, and the garbage classification detection is completed.
2. The method for detecting garbage categories using a neural network at a fixed distance according to claim 1, characterized in that The SSD target detection neural network based on deep learning is pre-trained by the following method: first, garbage pictures on the assembly line in the garbage treatment plant are collected; then the collected garbage pictures are labeled with garbage categories and locations through labeling software, and these labeled garbage pictures are used as data sets to train the SSD target detection neural network based on deep learning.
Citation Information
Patent Citations
Garbage classification-oriented multi-target detection method
CN111814860A
Information processing method and apparatus, and storage medium
WO2020248458A1